-- Logs begin at Fri 2022-07-01 18:45:03 UTC, end at Fri 2022-07-01 20:09:50 UTC. -- Jul 01 19:38:36.732537 dv-cinder-844601-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 01 19:38:36.739880 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 1 19:38:36 2022] *** Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: nodename: dv-cinder-844601-7-iscsi Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: machine: x86_64 Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: clock source: unix Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: pcre jit disabled Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: detected number of CPU cores: 4 Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: current working directory: / Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: detected binary path: /usr/bin/uwsgi-core Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: your processes number limit is 47680 Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: your memory page size is 4096 bytes Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: detected max file descriptor number: 2048 Jul 01 19:38:36.742965 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: lock engine: pthread robust mutexes Jul 01 19:38:36.743704 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: thunder lock: enabled Jul 01 19:38:36.743704 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 01 19:38:36.743704 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 01 19:38:36.788193 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: Python main interpreter initialized at 0x55f6550529c0 Jul 01 19:38:36.788193 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: python threads support enabled Jul 01 19:38:36.788193 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: your server socket listen backlog is limited to 100 connections Jul 01 19:38:36.788193 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: your mercy for graceful operations on workers is 90 seconds Jul 01 19:38:36.788557 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: mapped 403077 bytes (393 KB) for 2 cores Jul 01 19:38:36.788627 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: *** Operational MODE: preforking *** Jul 01 19:38:36.788886 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: *** uWSGI is running in multiple interpreter mode *** Jul 01 19:38:36.788886 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: spawned uWSGI master process (pid: 86768) Jul 01 19:38:36.789092 dv-cinder-844601-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 01 19:38:36.791911 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: spawned uWSGI worker 1 (pid: 86778, cores: 1) Jul 01 19:38:36.791911 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: spawned uWSGI worker 2 (pid: 86779, cores: 1) Jul 01 19:38:36.791911 dv-cinder-844601-7-iscsi devstack@keystone.service[86768]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 01 19:38:42.338366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 19:38:42.338366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: from cryptography.utils import int_from_bytes Jul 01 19:38:42.339159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 19:38:42.339159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: from cryptography.utils import int_from_bytes Jul 01 19:38:42.350308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 19:38:42.350308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: from cryptography.utils import int_from_bytes Jul 01 19:38:42.350771 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 19:38:42.350771 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: from cryptography.utils import int_from_bytes Jul 01 19:38:42.388087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 01 19:38:42.388277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 01 19:38:42.388470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] command line args: [] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 01 19:38:42.388741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 01 19:38:42.388918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ================================================================================ {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 01 19:38:42.389169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] admin_token = **** {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.389413 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] config_dir = [] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.389619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.389808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] config_source = [] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.389988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] debug = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.390185 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.390669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.390753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.390963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.391213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.391445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] list_limit = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.391627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] log_config_append = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.391805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.391982 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] log_dir = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.392160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] log_file = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.392408 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.392640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.393552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.394156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.394156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.394466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.394466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.394758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.394939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.395152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.395366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.395604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.395850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.396096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.396372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.396858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.397038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] publish_errors = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.397271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.397503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 01 19:38:42.397615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.397690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.397773 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 01 19:38:42.397909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] command line args: [] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 01 19:38:42.398064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.398192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.398310 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 01 19:38:42.398459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ================================================================================ {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 01 19:38:42.398579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] standard_threads = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.398579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.398786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] admin_token = **** {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.398907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.399026 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] config_dir = [] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.399168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.399287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.399399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] use_journal = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] config_source = [] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.399626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] use_json = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.399737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] debug = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.399847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] use_stderr = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.400003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.400150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] use_syslog = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.400274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.400392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.400531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.400645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.400760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.400878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.401000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.401118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.401236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] list_limit = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.401347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.401457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] log_config_append = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.401457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.401700 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.401813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] log_dir = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.401930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.402057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] log_file = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.402174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.402292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.402412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.402523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.402650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.402768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.402890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.403007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.403007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.403283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.403395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.403506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.403668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.403783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.403894 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.404005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.404119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.404242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.404361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.404473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.404589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.404701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.404812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.404930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.405042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.405152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.405267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.405385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.405497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.405645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.405645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.405864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.405864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] publish_errors = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.406083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.406196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.406306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.406417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.406527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.406637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.406755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.406872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.406984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.407094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.407204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.407314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] standard_threads = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.407432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.407545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.407656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.407766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.407883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.408000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.408000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] use_journal = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.408218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.408335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] use_json = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.409118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.409546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] use_stderr = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.409725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.409928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] use_syslog = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.409928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 19:38:42.410157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.410157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.410379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.410379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.410613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.410613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.410844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.410959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.411073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.411193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.411316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.411697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.411822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.411944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.412058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.412172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.412294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.412409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.412521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.412635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.412749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.412863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.412988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.413112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.413227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.413341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.413341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.413573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.413757 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.413878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.413993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.414107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.414221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.414335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.414458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.414573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.414687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.414802 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.414916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.415038 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.415160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.415274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.415394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.415509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.415623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.415811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.415937 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.416052 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.416167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.416280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.416394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.416519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.416641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.416756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.416870 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.416999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.417115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.417237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.417353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.417467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.417609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.417726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.417824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.417931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.418030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.418124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.418224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.418332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.418427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.418530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.418627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.418733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.418850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.418964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.419114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.419243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.419359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.419483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.419483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.419713 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.419828 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.419950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.420067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.420067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.420259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.420371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.420484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.420605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.420728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.420728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.420949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.421063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.421178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.421282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.421402 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.421518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.421640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.421756 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.421869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.421983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.422110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.422224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.422338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.422453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.422556 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.422668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.422817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.422940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.423056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.423170 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.423285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.423399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.423521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.423637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.423751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.423865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.423979 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.424093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.424214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.424331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.424444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.424567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.424682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.424796 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.424918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.425035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.425149 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.425281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.425399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.425524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.425701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.425820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.425934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.426056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.426171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.426292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.426417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.426534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.426648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.426763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] auth.external = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.426877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.426992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.427177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.427295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.427419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.427535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.427649 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.427763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] auth.password = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.427886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.428011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] auth.token = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.428139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.428255 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.428369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.428485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.428598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.428719 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.428843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.428960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.429074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.429189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.429304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.429427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.429548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.429681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.429796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.429910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] credential.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.430035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.430161 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.430277 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.430392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.430513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.430630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.430743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.430866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.430981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.431096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.431210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.431324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.431437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.431581 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.431706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.431834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.431951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.432066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.432188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.432304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.432418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.432542 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.432660 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.432775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.432897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.432897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.433123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.433248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.433365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.433480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.433609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.433720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.433834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] federation.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.433965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.434090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.434207 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.434322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.434436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.434551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.434551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.434778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.434894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.435009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.435131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.435247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] auth.external = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.435247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.435479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.435479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.435697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.435814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.435939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.436067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.436184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] auth.password = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.436299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.436414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] auth.token = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.436530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.436645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.436768 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.436886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.437002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.437117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.437232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.437348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.437480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.437650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.437770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.437888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.438004 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.438118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.438246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.438364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] credential.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.438479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.438594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.438709 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.438824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.438949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.439065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.439065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.439297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.439414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.439533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.439656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.439773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.439887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.440003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.440118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.440238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.440362 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.440478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.440594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.440708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.440831 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.441022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.441143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.441258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.441258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.441481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.441612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.441742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.441861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.441976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.442093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.442208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] federation.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.442323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.442446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.442563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.442678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.442793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.442908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.443023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.443147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.443264 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.443379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.443495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.443609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.443725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.443848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.443966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.444089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.444207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.444322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.444444 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.444571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.444688 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.444803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.444919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.445034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.445149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.445273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.445396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.445512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.445652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.445769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.445885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.446011 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.446129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.446244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.446360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.446474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.446589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.446712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.446828 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.446943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.447059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.447174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.447288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.447416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.447540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.447656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.447771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.447886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.448000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.448123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.448240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.448356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.448472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.448587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.448701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.448824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.448941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.449056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.449171 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.449286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.449401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.449528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.449655 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.449771 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.449886 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.450001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.450115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.450242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.450359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.450475 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.450590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.450704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.450819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.450942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.451059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.451059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.451275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.451389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.451505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.451628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.451744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.451859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.451973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.452088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.452202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.452325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.452441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.452557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.452672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.452786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.452909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.453025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.453140 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.453254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.453368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.453483 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.453665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.453785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.453900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.454015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.454130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.454244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.454367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.454599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.454715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.454830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.454944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.455067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.455184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.455793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.455910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.456025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.456140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.456263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.456381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.456496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.456611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.456727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.456842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.456966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.457083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.457198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.457313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.457428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.457542 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.457687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.457807 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] resource.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.457923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.457923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.458140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.458256 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.458380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.458496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.458611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.458726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.458843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.458957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.459084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.459201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.459316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.459431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.459550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.459665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.459789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.459907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.460022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.460136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.460252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] role.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.460366 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.460490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] role.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.460606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.460722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.460836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.460953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.461068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.461192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.461309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.461425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.461540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.461671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.461787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.461914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.462033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.462149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.462264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.462379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.462494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.462617 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.462734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.462849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.462964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.463081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.463179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.463284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.463399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.463514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.463629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.463744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.463859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.463859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.464227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.464227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.464449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.464449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.464679 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.464797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.464912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.465027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.465143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.465258 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.465381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.465498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.465668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.465786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.465901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.466016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.466142 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.466262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] resource.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.466377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.466492 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.466607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.466722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.466845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.466845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.467062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.467176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.467292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.467407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.467530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.467530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.467747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.467747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.467962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.468076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.468199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.468317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.468432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.468546 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] token.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.468661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] role.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.468775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.468898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] role.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.469014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.469129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.469243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.469359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.469477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.469625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.469746 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.469861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.469976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.470091 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.470206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.470329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.470446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.470561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.470676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.470791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.471485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.471623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.471742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.471857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.471972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.472087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.472203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.472326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.472443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.472559 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.472674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.472789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.472904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.473028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.473145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.473299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.473420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.connection = **** {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.473546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.473715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.473833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.473957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.474075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.474190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.474305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.474421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.474536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.474778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.474893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.475008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.475123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.475238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.475362 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.475479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.475595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.475711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.475825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.475940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.476067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.476184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.476299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.476414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.476615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.476737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.476862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.476995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.477110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.477231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.477347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.477462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.477603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.477723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] token.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.477838 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.477953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.478068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86779) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 01 19:38:42.478183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.478183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.478183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.478183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.478183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.478183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.478183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.478183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.478183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.479124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.479124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.479124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.479124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.479552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.479552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.479769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.479954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.connection = **** {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.480203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.480457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.480701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.480948 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.481194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.481436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.481702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.481943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.482191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.482429 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.482675 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.482913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.483164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.483402 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.483649 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.483882 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.484125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.484360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 19:38:42.484531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86778) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 01 19:38:42.506117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.506346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.506571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.510059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.510292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.510516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.515029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.519453 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.544262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.544495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.546168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.546405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.690036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.690222 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.740201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.740431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.740756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.740994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.805488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.805700 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.805794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.805938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.806043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.806160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.806243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.806379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.806460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.806593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.806674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.806806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.830982 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.831176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.831763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.831897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.831980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.832112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.832192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.832326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.832409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.832538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.832620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.832750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.832838 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.832965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.843890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.844180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.856593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.856840 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.857079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.857163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.857301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.857383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.857517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.857708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.857788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.857859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.857966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86779) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.858048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.858182 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.858400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86778) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 19:38:42.875416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.875628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 19:38:42.875962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.876046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.876157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 19:38:42.876241 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 19:38:42.876422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.876504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.876674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 19:38:42.876754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.876926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.877009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.877730 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.877823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.878018 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.878233 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.888599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.888873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.889040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.889147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.889314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.889431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.889605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.889878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.898252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.898731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.901521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.901826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.902107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.902385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.902468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.902662 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.902747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.902915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.902999 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.903178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.903268 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.903444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.903527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.903709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.903790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.903977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.904238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.904321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.904499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.904771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.905036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.932006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.932806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.933628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.933908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.934434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.934709 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.939194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.939475 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.939783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.939947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.940392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.940744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.947197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.947471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.947995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.948273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.951965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.952249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.952526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.952780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.952883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.953064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.953154 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.953332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.953421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.953617 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.953722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.953885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.953975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.954153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.954414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.954674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.973536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.973826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.974181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.974454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.978431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.979131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.982139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.982431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.982776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.983063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.988213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.988976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.991850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.992578 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:42.995509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.995806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.996090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.996250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.996378 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.996540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.996646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.996822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.996934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.997101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.997385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:42.997690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.016005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.016285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.016550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.016810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.016906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.017138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.017401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.017690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.024819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.025727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.026574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.026847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.027429 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.027711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.030072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.030358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.030641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.030933 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.031013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.031214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.031304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.031488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.031758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.032052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.045476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.045778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.046067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.046164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.046344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.046428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.046620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.046703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.046891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.046974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.047165 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.047257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.048703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.049091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.066952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.067344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.070232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.070628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.073558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.073950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.075166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.075685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.076936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.077213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.077316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.077501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.077598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.077892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.085291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.085843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.087099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.087454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.090446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.090703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.090786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.090997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.091080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.091291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.091386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.091646 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.101031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.101318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.101401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.101621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.101709 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.101973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.109440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86778) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.109747 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.109845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86779) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 19:38:43.110049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.110140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.110396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.110580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.110672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.110847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.110939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.111118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.111202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.111382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.111534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.111810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.111906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.112096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.112182 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.112372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.112454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.112648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.112738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.112909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.112991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.113172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.113254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86778) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.113435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86779) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 19:38:43.151474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f6550529c0 pid: 86778 (default app) Jul 01 19:38:43.151768 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f6550529c0 pid: 86779 (default app) Jul 01 19:38:43.154871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccc447-57f0-4716-a964-84514ce0857c None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:43.155167 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccc447-57f0-4716-a964-84514ce0857c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:43.155447 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccc447-57f0-4716-a964-84514ce0857c None None] PATH_INFO: `/v3` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:43.160349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1/1] 10.222.0.66 () {50 vars in 791 bytes} [Fri Jul 1 19:38:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:38:50.951940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf1fca-ffc9-4a88-9332-da72a6a76050 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:50.952492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf1fca-ffc9-4a88-9332-da72a6a76050 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:50.952492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf1fca-ffc9-4a88-9332-da72a6a76050 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:50.957530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1/2] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:38:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:38:50.962948 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:50.963390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:50.963484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:51.034097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: INFO dbcounter [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] Registered counter for database keystone Jul 01 19:38:51.055352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG oslo_db.sqlalchemy.engines [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86778) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 01 19:38:51.061776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writer thread running {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 01 19:38:51.236075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG passlib.handlers.bcrypt [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86778) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 01 19:38:51.236447 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG passlib.handlers.bcrypt [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86778) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 01 19:38:51.291109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:38:51.292707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:38:51.414755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 2/3] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:38:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 01 19:38:51.620819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf1fca-ffc9-4a88-9332-da72a6a76050 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:51.692609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: INFO dbcounter [None req-b2cf1fca-ffc9-4a88-9332-da72a6a76050 None None] Registered counter for database keystone Jul 01 19:38:51.711640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG oslo_db.sqlalchemy.engines [None req-b2cf1fca-ffc9-4a88-9332-da72a6a76050 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86779) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 01 19:38:51.717870 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writer thread running {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 01 19:38:51.838010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf1fca-ffc9-4a88-9332-da72a6a76050 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:51.842315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1fa82-063f-40f4-8c9c-1041cc0876a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:51.842967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1fa82-063f-40f4-8c9c-1041cc0876a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:51.843250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1fa82-063f-40f4-8c9c-1041cc0876a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:51.843524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1fa82-063f-40f4-8c9c-1041cc0876a2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:51.852133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1fa82-063f-40f4-8c9c-1041cc0876a2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:51.904361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: WARNING keystone.common.rbac_enforcer.enforcer [None req-58c1fa82-063f-40f4-8c9c-1041cc0876a2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 01 19:38:51.904707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1fa82-063f-40f4-8c9c-1041cc0876a2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:51.915927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 2/4] 10.222.0.66 () {58 vars in 1324 bytes} [Fri Jul 1 19:38:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:38:51.922101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:51.937438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab20f78-6df3-48f5-9de2-20a427c697f2 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:51.942402 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288b99d-4c93-4dfa-bd03-bdb48b04a0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:51.943511 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288b99d-4c93-4dfa-bd03-bdb48b04a0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:51.943793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288b99d-4c93-4dfa-bd03-bdb48b04a0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:51.944062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288b99d-4c93-4dfa-bd03-bdb48b04a0c7 admin admin] PATH_INFO: `/v3/projects/54a2d5d499f64a6a8437120d4d4f1f46` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:51.945531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288b99d-4c93-4dfa-bd03-bdb48b04a0c7 admin admin] RBAC: Authorizing `identity:get_project(project_id=54a2d5d499f64a6a8437120d4d4f1f46)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:52.002939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f288b99d-4c93-4dfa-bd03-bdb48b04a0c7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 01 19:38:52.003496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288b99d-4c93-4dfa-bd03-bdb48b04a0c7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:52.007863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 3/5] 10.222.0.66 () {56 vars in 1207 bytes} [Fri Jul 1 19:38:51 2022] GET /identity/v3/projects/54a2d5d499f64a6a8437120d4d4f1f46 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:38:54.054616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25af47-f7b2-41cc-a68f-746895d3c3c1 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:54.055125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25af47-f7b2-41cc-a68f-746895d3c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:54.055226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25af47-f7b2-41cc-a68f-746895d3c3c1 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:54.059278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 3/6] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:38:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:38:54.065179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0049e2d-3a31-4d16-b573-ec62942b4bb7 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:54.065558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0049e2d-3a31-4d16-b573-ec62942b4bb7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:54.065760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0049e2d-3a31-4d16-b573-ec62942b4bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:54.085110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f0049e2d-3a31-4d16-b573-ec62942b4bb7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:38:54.086690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f0049e2d-3a31-4d16-b573-ec62942b4bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:38:54.094495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 4/7] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:38:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 01 19:38:54.117072 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25af47-f7b2-41cc-a68f-746895d3c3c1 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:54.129464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25af47-f7b2-41cc-a68f-746895d3c3c1 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:54.133686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33260ed2-0b09-4d26-90a0-c3adbbdd9809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:54.134315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33260ed2-0b09-4d26-90a0-c3adbbdd9809 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:54.134594 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33260ed2-0b09-4d26-90a0-c3adbbdd9809 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:54.134867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33260ed2-0b09-4d26-90a0-c3adbbdd9809 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:54.143418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33260ed2-0b09-4d26-90a0-c3adbbdd9809 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:54.145261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33260ed2-0b09-4d26-90a0-c3adbbdd9809 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:54.153956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 4/8] 10.222.0.66 () {58 vars in 1324 bytes} [Fri Jul 1 19:38:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 19:38:54.159210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0049e2d-3a31-4d16-b573-ec62942b4bb7 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:54.168488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0049e2d-3a31-4d16-b573-ec62942b4bb7 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:54.172825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f9543-aa82-4336-88d8-b3ec04e7ea92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:54.173729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f9543-aa82-4336-88d8-b3ec04e7ea92 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:54.174005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f9543-aa82-4336-88d8-b3ec04e7ea92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:54.174276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f9543-aa82-4336-88d8-b3ec04e7ea92 admin admin] PATH_INFO: `/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:54.175663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f9543-aa82-4336-88d8-b3ec04e7ea92 admin admin] RBAC: Authorizing `identity:get_user(user_id=9d09bdd9927e44ffbfd9b2ec447a5887)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:54.177479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f9543-aa82-4336-88d8-b3ec04e7ea92 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:54.179066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 5/9] 10.222.0.66 () {56 vars in 1198 bytes} [Fri Jul 1 19:38:54 2022] GET /identity/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:38:56.307734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ebfed-74ec-4949-b2b8-0afaf5acd4e3 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:56.308284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ebfed-74ec-4949-b2b8-0afaf5acd4e3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:56.308284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ebfed-74ec-4949-b2b8-0afaf5acd4e3 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:56.309788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 5/10] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:38:56.314726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105294c-e27e-49e0-81d1-8610b30814ea None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:56.315753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105294c-e27e-49e0-81d1-8610b30814ea None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:56.315753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105294c-e27e-49e0-81d1-8610b30814ea None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:56.315911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15326c6-ebc2-4d7d-899f-b4b7c2bfcb49 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:56.315911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15326c6-ebc2-4d7d-899f-b4b7c2bfcb49 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:56.316070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15326c6-ebc2-4d7d-899f-b4b7c2bfcb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:56.316708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 6/11] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:38:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:38:56.321526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3c0b-cdba-4fd9-8eaf-9a8170abcad5 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:56.321853 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3c0b-cdba-4fd9-8eaf-9a8170abcad5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:56.322138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3c0b-cdba-4fd9-8eaf-9a8170abcad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:56.338890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-104e3c0b-cdba-4fd9-8eaf-9a8170abcad5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:38:56.341161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-104e3c0b-cdba-4fd9-8eaf-9a8170abcad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:38:56.356029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG passlib.handlers.bcrypt [None req-f15326c6-ebc2-4d7d-899f-b4b7c2bfcb49 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86779) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 01 19:38:56.356438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG passlib.handlers.bcrypt [None req-f15326c6-ebc2-4d7d-899f-b4b7c2bfcb49 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86779) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 01 19:38:56.402383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f15326c6-ebc2-4d7d-899f-b4b7c2bfcb49 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:38:56.403971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f15326c6-ebc2-4d7d-899f-b4b7c2bfcb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:38:56.422742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 7/12] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:38:56.445562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3c0b-cdba-4fd9-8eaf-9a8170abcad5 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:56.469920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:38:56.506206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 6/13] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:38:56.529342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15326c6-ebc2-4d7d-899f-b4b7c2bfcb49 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:56.554513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3c0b-cdba-4fd9-8eaf-9a8170abcad5 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:56.589707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecbbce-1c3a-42f2-beda-c1631b12956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:56.590362 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecbbce-1c3a-42f2-beda-c1631b12956a None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:56.590642 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecbbce-1c3a-42f2-beda-c1631b12956a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:56.590914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecbbce-1c3a-42f2-beda-c1631b12956a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:56.631966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15326c6-ebc2-4d7d-899f-b4b7c2bfcb49 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:56.634255 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:38:56.660377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecbbce-1c3a-42f2-beda-c1631b12956a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:56.662172 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecbbce-1c3a-42f2-beda-c1631b12956a None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:56.669469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95daae-5b01-47b5-ba3f-6d75aaa718db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:56.670529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95daae-5b01-47b5-ba3f-6d75aaa718db None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:56.670813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95daae-5b01-47b5-ba3f-6d75aaa718db None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:56.671084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95daae-5b01-47b5-ba3f-6d75aaa718db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:56.683257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95daae-5b01-47b5-ba3f-6d75aaa718db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:38:56.685169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-5e95daae-5b01-47b5-ba3f-6d75aaa718db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:38:56.695405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:38:56.696321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:38:56.697099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 7/14] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:38:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:38:56.702344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95daae-5b01-47b5-ba3f-6d75aaa718db None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:56.762931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 8/15] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:38:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 19:38:56.769322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecbbce-1c3a-42f2-beda-c1631b12956a None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:56.808272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95daae-5b01-47b5-ba3f-6d75aaa718db None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:56.843903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a6384-4821-4ecc-8f73-6aa045764ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:56.844516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a6384-4821-4ecc-8f73-6aa045764ff4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:56.844804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a6384-4821-4ecc-8f73-6aa045764ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:56.845085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a6384-4821-4ecc-8f73-6aa045764ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:56.846766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a6384-4821-4ecc-8f73-6aa045764ff4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:38:56.847078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a6384-4821-4ecc-8f73-6aa045764ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:56.848842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a6384-4821-4ecc-8f73-6aa045764ff4 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:56.860489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 8/16] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:38:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:38:56.866293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a6384-4821-4ecc-8f73-6aa045764ff4 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:56.868890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecbbce-1c3a-42f2-beda-c1631b12956a None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:56.891610 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:38:56.896094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:38:56.904029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f366d43-63c8-4083-a294-b658c59140ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:56.904828 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f366d43-63c8-4083-a294-b658c59140ee None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:56.905103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f366d43-63c8-4083-a294-b658c59140ee None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:56.905370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f366d43-63c8-4083-a294-b658c59140ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:56.916943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f366d43-63c8-4083-a294-b658c59140ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:56.918461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f366d43-63c8-4083-a294-b658c59140ee None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-2f366d43-63c8-4083-a294-b658c59140ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:38:56.939312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:38:56.940152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:38:56.940941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:38:56.940941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:38:56.940941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:38:56.940941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:38:56.940941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 9/17] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:38:56.946381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f366d43-63c8-4083-a294-b658c59140ee None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:56.968237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a6384-4821-4ecc-8f73-6aa045764ff4 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:57.001417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a990d5-33c3-4409-b195-a40ea31e72c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:57.002873 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a990d5-33c3-4409-b195-a40ea31e72c2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:57.003181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a990d5-33c3-4409-b195-a40ea31e72c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:57.003449 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a990d5-33c3-4409-b195-a40ea31e72c2 None admin] PATH_INFO: `/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:57.005011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a990d5-33c3-4409-b195-a40ea31e72c2 None admin] RBAC: Authorizing `identity:get_user(user_id=9d09bdd9927e44ffbfd9b2ec447a5887)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:57.006510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a990d5-33c3-4409-b195-a40ea31e72c2 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:57.008151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 9/18] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:38:56 2022] GET /identity/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:38:57.013381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a990d5-33c3-4409-b195-a40ea31e72c2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:57.046852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f366d43-63c8-4083-a294-b658c59140ee None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:57.080884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ccdde-d2ea-4e7c-942e-f6dcffd3e756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:57.081550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ccdde-d2ea-4e7c-942e-f6dcffd3e756 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:57.081875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ccdde-d2ea-4e7c-942e-f6dcffd3e756 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:57.082146 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ccdde-d2ea-4e7c-942e-f6dcffd3e756 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:57.082877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ccdde-d2ea-4e7c-942e-f6dcffd3e756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:38:57.083186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ccdde-d2ea-4e7c-942e-f6dcffd3e756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:57.084581 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ccdde-d2ea-4e7c-942e-f6dcffd3e756 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:57.102268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 10/19] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:38:57.105752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:38:57.113183 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a990d5-33c3-4409-b195-a40ea31e72c2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:57.151537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed9aab-1fc0-4e93-a983-85f259d88dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:57.152354 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed9aab-1fc0-4e93-a983-85f259d88dd8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:57.152640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed9aab-1fc0-4e93-a983-85f259d88dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:57.152918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed9aab-1fc0-4e93-a983-85f259d88dd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:57.154238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed9aab-1fc0-4e93-a983-85f259d88dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:57.155759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed9aab-1fc0-4e93-a983-85f259d88dd8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:57.157703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 10/20] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:38:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:38:57.163944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ccdde-d2ea-4e7c-942e-f6dcffd3e756 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:57.200466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:38:57.271957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ccdde-d2ea-4e7c-942e-f6dcffd3e756 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:57.306045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c44ce-1eb7-4086-a0a6-78a1f95ca9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:57.306709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c44ce-1eb7-4086-a0a6-78a1f95ca9cd None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:57.306989 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c44ce-1eb7-4086-a0a6-78a1f95ca9cd None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:57.307263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c44ce-1eb7-4086-a0a6-78a1f95ca9cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:57.308103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c44ce-1eb7-4086-a0a6-78a1f95ca9cd None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.domain.id=default, user.id=9d09bdd9927e44ffbfd9b2ec447a5887) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:38:57.308408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c44ce-1eb7-4086-a0a6-78a1f95ca9cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:57.309879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c44ce-1eb7-4086-a0a6-78a1f95ca9cd None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:57.317491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 11/21] 10.222.0.66 () {56 vars in 1322 bytes} [Fri Jul 1 19:38:57 2022] GET /identity/v3/role_assignments?user.id=9d09bdd9927e44ffbfd9b2ec447a5887&scope.domain.id=default&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:38:59.531743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5581e1-2d41-459c-8e85-947d8b6648a2 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:59.532753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5581e1-2d41-459c-8e85-947d8b6648a2 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:59.534161 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5581e1-2d41-459c-8e85-947d8b6648a2 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:59.536099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 11/22] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:38:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:38:59.541438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad177b67-4410-4392-8601-ff71b001a3d1 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:59.541886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad177b67-4410-4392-8601-ff71b001a3d1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:59.542057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad177b67-4410-4392-8601-ff71b001a3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:59.571439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ad177b67-4410-4392-8601-ff71b001a3d1 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:38:59.572653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ad177b67-4410-4392-8601-ff71b001a3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:38:59.665891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 12/23] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:38:59.702616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5581e1-2d41-459c-8e85-947d8b6648a2 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:38:59.816853 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3cd4b-d3b3-40ff-ac04-7af9b2af2066 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:59.817526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3cd4b-d3b3-40ff-ac04-7af9b2af2066 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:59.818031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3cd4b-d3b3-40ff-ac04-7af9b2af2066 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:59.819616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 13/24] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:38:59.824863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf8160-13ba-4530-981f-f5f9f2bebef2 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:59.825501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf8160-13ba-4530-981f-f5f9f2bebef2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:59.826030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf8160-13ba-4530-981f-f5f9f2bebef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:59.835428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5581e1-2d41-459c-8e85-947d8b6648a2 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:38:59.851791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-77cf8160-13ba-4530-981f-f5f9f2bebef2 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:38:59.853480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-77cf8160-13ba-4530-981f-f5f9f2bebef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:38:59.885190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68c48e-cefb-4c28-8ac2-993a8273637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:38:59.889150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68c48e-cefb-4c28-8ac2-993a8273637a None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:59.889150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68c48e-cefb-4c28-8ac2-993a8273637a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:59.889150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68c48e-cefb-4c28-8ac2-993a8273637a None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:59.889150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68c48e-cefb-4c28-8ac2-993a8273637a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:38:59.889150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68c48e-cefb-4c28-8ac2-993a8273637a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:38:59.893966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:38:59.984762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 14/25] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:38:59.986806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b36fa4-382a-4a94-a527-a52dd631625c None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:59.987098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b36fa4-382a-4a94-a527-a52dd631625c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:59.987373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b36fa4-382a-4a94-a527-a52dd631625c None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:38:59.992738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 15/26] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:38:59.994009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6bbbf-c689-4c22-8ef8-8ed5e86156f8 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:38:59.994296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6bbbf-c689-4c22-8ef8-8ed5e86156f8 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:38:59.994574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6bbbf-c689-4c22-8ef8-8ed5e86156f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:00.026665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9ef6bbbf-c689-4c22-8ef8-8ed5e86156f8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:00.027872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9ef6bbbf-c689-4c22-8ef8-8ed5e86156f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:00.055582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 12/27] 10.222.0.66 () {60 vars in 1131 bytes} [Fri Jul 1 19:38:59 2022] POST /identity/v3/roles => generated 219 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:00.057395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a17b-ca8e-449d-8edf-17578964f63e None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:00.057872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a17b-ca8e-449d-8edf-17578964f63e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:00.058153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a17b-ca8e-449d-8edf-17578964f63e None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:00.059420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 13/28] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:00.063522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9a17b-ca8e-449d-8edf-17578964f63e None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:00.102356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:00.117417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 16/29] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:00.119737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07c869-4e6b-4bbe-a161-b46fe48161b1 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:00.120227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07c869-4e6b-4bbe-a161-b46fe48161b1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:00.120696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07c869-4e6b-4bbe-a161-b46fe48161b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:00.144802 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ac07c869-4e6b-4bbe-a161-b46fe48161b1 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:00.146531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ac07c869-4e6b-4bbe-a161-b46fe48161b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:00.209012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9a17b-ca8e-449d-8edf-17578964f63e None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:00.233798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 17/30] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:00.235410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07c869-4e6b-4bbe-a161-b46fe48161b1 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:00.268900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:00.328439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5661f26-5a81-4a22-9ea4-97066789e8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:00.329250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5661f26-5a81-4a22-9ea4-97066789e8cf None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:00.329530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5661f26-5a81-4a22-9ea4-97066789e8cf None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:00.330448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5661f26-5a81-4a22-9ea4-97066789e8cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:00.357081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5661f26-5a81-4a22-9ea4-97066789e8cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-a5661f26-5a81-4a22-9ea4-97066789e8cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:00.365715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:00.366539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:00.367287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:00.369664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 14/31] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:00.370521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5661f26-5a81-4a22-9ea4-97066789e8cf None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:00.402713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07c869-4e6b-4bbe-a161-b46fe48161b1 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:00.454306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cd20a-583c-45d5-9c48-e1d6a9d7ad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:00.455627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cd20a-583c-45d5-9c48-e1d6a9d7ad57 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:00.455627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cd20a-583c-45d5-9c48-e1d6a9d7ad57 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:00.455981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cd20a-583c-45d5-9c48-e1d6a9d7ad57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:00.479503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:00.484820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cd20a-583c-45d5-9c48-e1d6a9d7ad57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:00.487288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cd20a-583c-45d5-9c48-e1d6a9d7ad57 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:00.499869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5661f26-5a81-4a22-9ea4-97066789e8cf None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-253cd20a-583c-45d5-9c48-e1d6a9d7ad57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:39:00.511112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:00.512168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:00.513153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:39:00.513153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:39:00.513153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:39:00.513153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:00.513153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 18/32] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:00.513153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cd20a-583c-45d5-9c48-e1d6a9d7ad57 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:00.544786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41d700-684d-4f0b-a83d-5e4b330275f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:00.545880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41d700-684d-4f0b-a83d-5e4b330275f7 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:00.546243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41d700-684d-4f0b-a83d-5e4b330275f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:00.546602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41d700-684d-4f0b-a83d-5e4b330275f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:00.547439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41d700-684d-4f0b-a83d-5e4b330275f7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:00.549694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41d700-684d-4f0b-a83d-5e4b330275f7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:00.569658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 15/33] 10.222.0.66 () {60 vars in 1131 bytes} [Fri Jul 1 19:39:00 2022] POST /identity/v3/roles => generated 225 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:00.574357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41d700-684d-4f0b-a83d-5e4b330275f7 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:00.641525 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:00.721179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cd20a-583c-45d5-9c48-e1d6a9d7ad57 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:00.759153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41d700-684d-4f0b-a83d-5e4b330275f7 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:00.787642 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77599f-be6d-4db4-9361-5dc2d4718803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:00.788329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77599f-be6d-4db4-9361-5dc2d4718803 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:00.788612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77599f-be6d-4db4-9361-5dc2d4718803 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:00.788888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77599f-be6d-4db4-9361-5dc2d4718803 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:00.789694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77599f-be6d-4db4-9361-5dc2d4718803 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:00.790558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77599f-be6d-4db4-9361-5dc2d4718803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:00.792259 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77599f-be6d-4db4-9361-5dc2d4718803 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:00.806798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492ba8c-7901-4821-a4d7-68f94bed2f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:00.807521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492ba8c-7901-4821-a4d7-68f94bed2f89 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:00.807812 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492ba8c-7901-4821-a4d7-68f94bed2f89 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:00.808097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492ba8c-7901-4821-a4d7-68f94bed2f89 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:00.808477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 19/34] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:00.808904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492ba8c-7901-4821-a4d7-68f94bed2f89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:00.809219 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492ba8c-7901-4821-a4d7-68f94bed2f89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:00.810681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492ba8c-7901-4821-a4d7-68f94bed2f89 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:00.814941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77599f-be6d-4db4-9361-5dc2d4718803 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:00.838299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 16/35] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:00.844152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492ba8c-7901-4821-a4d7-68f94bed2f89 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:00.847903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:00.951608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77599f-be6d-4db4-9361-5dc2d4718803 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:00.979521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492ba8c-7901-4821-a4d7-68f94bed2f89 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:00.989862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:00.997145 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011ad9d-8ad9-47f8-a3fa-de0683e9a336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:00.998345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011ad9d-8ad9-47f8-a3fa-de0683e9a336 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:00.998629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011ad9d-8ad9-47f8-a3fa-de0683e9a336 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:00.998907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011ad9d-8ad9-47f8-a3fa-de0683e9a336 None admin] PATH_INFO: `/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.000181 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011ad9d-8ad9-47f8-a3fa-de0683e9a336 None admin] RBAC: Authorizing `identity:get_user(user_id=9d09bdd9927e44ffbfd9b2ec447a5887)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:01.002531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011ad9d-8ad9-47f8-a3fa-de0683e9a336 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:01.005046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 20/36] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:00 2022] GET /identity/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:01.010798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011ad9d-8ad9-47f8-a3fa-de0683e9a336 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:01.038751 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef2062-af6b-4c34-8396-143fd76cf640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:01.039698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef2062-af6b-4c34-8396-143fd76cf640 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:01.039979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef2062-af6b-4c34-8396-143fd76cf640 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:01.040495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef2062-af6b-4c34-8396-143fd76cf640 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.041723 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef2062-af6b-4c34-8396-143fd76cf640 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:01.043555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef2062-af6b-4c34-8396-143fd76cf640 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:01.087130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 17/37] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:39:00 2022] POST /identity/v3/projects => generated 301 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:01.088452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf069c-d65b-4c7f-a57c-5eef1c4fb014 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:01.088731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf069c-d65b-4c7f-a57c-5eef1c4fb014 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:01.089006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf069c-d65b-4c7f-a57c-5eef1c4fb014 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.090282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 18/38] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:01.097117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413869b7-8962-4ca6-b3bc-b0e431bcc9cb None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:01.097552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413869b7-8962-4ca6-b3bc-b0e431bcc9cb None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:01.097963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413869b7-8962-4ca6-b3bc-b0e431bcc9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.121596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-413869b7-8962-4ca6-b3bc-b0e431bcc9cb None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:01.124481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-413869b7-8962-4ca6-b3bc-b0e431bcc9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:01.159870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011ad9d-8ad9-47f8-a3fa-de0683e9a336 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:01.191112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:01.213269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142d336-64e5-45ec-8fc6-392d8e80d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:01.213969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 19/39] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:01.214620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142d336-64e5-45ec-8fc6-392d8e80d175 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:01.215154 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142d336-64e5-45ec-8fc6-392d8e80d175 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:01.215563 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142d336-64e5-45ec-8fc6-392d8e80d175 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.216987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142d336-64e5-45ec-8fc6-392d8e80d175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:01.218641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142d336-64e5-45ec-8fc6-392d8e80d175 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:01.221033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 21/40] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:01.229608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413869b7-8962-4ca6-b3bc-b0e431bcc9cb None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:01.240530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142d336-64e5-45ec-8fc6-392d8e80d175 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:01.328382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:01.352479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413869b7-8962-4ca6-b3bc-b0e431bcc9cb None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:01.362608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142d336-64e5-45ec-8fc6-392d8e80d175 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:01.388191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f566c-5f04-494a-b305-a4226ddf1889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:01.389105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f566c-5f04-494a-b305-a4226ddf1889 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:01.389400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f566c-5f04-494a-b305-a4226ddf1889 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:01.389721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f566c-5f04-494a-b305-a4226ddf1889 None admin] PATH_INFO: `/v3/domains/default/users/9d09bdd9927e44ffbfd9b2ec447a5887/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.391777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f566c-5f04-494a-b305-a4226ddf1889 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9d09bdd9927e44ffbfd9b2ec447a5887, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:01.393290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f566c-5f04-494a-b305-a4226ddf1889 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:01.396582 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb66d33-a24b-4a67-9c87-a22ead48de9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:01.397264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb66d33-a24b-4a67-9c87-a22ead48de9c None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:01.397545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb66d33-a24b-4a67-9c87-a22ead48de9c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:01.397859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb66d33-a24b-4a67-9c87-a22ead48de9c None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.398658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb66d33-a24b-4a67-9c87-a22ead48de9c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:01.400030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb66d33-a24b-4a67-9c87-a22ead48de9c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:01.407155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 20/41] 10.222.0.66 () {58 vars in 1361 bytes} [Fri Jul 1 19:39:01 2022] PUT /identity/v3/domains/default/users/9d09bdd9927e44ffbfd9b2ec447a5887/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:01.409619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161ae58-836c-4924-8c3f-d36e0d99722d None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:01.409896 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161ae58-836c-4924-8c3f-d36e0d99722d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:01.410165 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161ae58-836c-4924-8c3f-d36e0d99722d None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.412090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 21/42] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:01.417836 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453ff75-0f7c-4423-b21a-6d1c19c44385 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:01.418707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453ff75-0f7c-4423-b21a-6d1c19c44385 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:01.419189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453ff75-0f7c-4423-b21a-6d1c19c44385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.438895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-0453ff75-0f7c-4423-b21a-6d1c19c44385 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:01.441240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0453ff75-0f7c-4423-b21a-6d1c19c44385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:01.491477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:01.530865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 22/43] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:01.558502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453ff75-0f7c-4423-b21a-6d1c19c44385 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:01.630592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 22/44] 10.222.0.66 () {60 vars in 1131 bytes} [Fri Jul 1 19:39:01 2022] POST /identity/v3/roles => generated 223 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:39:01.681156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453ff75-0f7c-4423-b21a-6d1c19c44385 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:01.719475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abad2c7-35dd-4349-84bb-441f991098ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:01.720402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abad2c7-35dd-4349-84bb-441f991098ee None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:01.720683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abad2c7-35dd-4349-84bb-441f991098ee None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:01.720955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abad2c7-35dd-4349-84bb-441f991098ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.722411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abad2c7-35dd-4349-84bb-441f991098ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:01.723912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abad2c7-35dd-4349-84bb-441f991098ee None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:01.725788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 23/45] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:01.731452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb66d33-a24b-4a67-9c87-a22ead48de9c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:01.841336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb66d33-a24b-4a67-9c87-a22ead48de9c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:01.868076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:01.876379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb9f88-2e7b-40ff-a82e-3528643ed846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:01.877623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb9f88-2e7b-40ff-a82e-3528643ed846 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:01.878005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb9f88-2e7b-40ff-a82e-3528643ed846 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:01.878285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb9f88-2e7b-40ff-a82e-3528643ed846 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:01.878998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb9f88-2e7b-40ff-a82e-3528643ed846 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:01.880569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb9f88-2e7b-40ff-a82e-3528643ed846 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:01.906255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 23/46] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:39:01 2022] POST /identity/v3/projects => generated 312 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:03.501622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db687579-a047-406e-946d-33529873db31 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:03.501622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db687579-a047-406e-946d-33529873db31 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:03.501622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db687579-a047-406e-946d-33529873db31 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:03.503763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 24/47] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:03.509237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ec28b-b358-466d-b104-06b08026ac9b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:03.509696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ec28b-b358-466d-b104-06b08026ac9b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:03.509836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ec28b-b358-466d-b104-06b08026ac9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:03.527527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-2f9ec28b-b358-466d-b104-06b08026ac9b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:03.529048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2f9ec28b-b358-466d-b104-06b08026ac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:03.601334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 24/48] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:03.625022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db687579-a047-406e-946d-33529873db31 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:03.669549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:03.728566 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db687579-a047-406e-946d-33529873db31 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:03.762569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce68c38-b69e-4904-8bbf-4bb2557f7119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:03.763290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce68c38-b69e-4904-8bbf-4bb2557f7119 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:03.763575 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce68c38-b69e-4904-8bbf-4bb2557f7119 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:03.763851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce68c38-b69e-4904-8bbf-4bb2557f7119 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:03.774516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce68c38-b69e-4904-8bbf-4bb2557f7119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-cce68c38-b69e-4904-8bbf-4bb2557f7119 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:03.782224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:03.783261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:03.784166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 25/49] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:03.789634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ec28b-b358-466d-b104-06b08026ac9b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:03.887571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ec28b-b358-466d-b104-06b08026ac9b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:03.898094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:03.923353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fee2e2-c48d-4cd7-b4b6-a06a971f84a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:03.924663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fee2e2-c48d-4cd7-b4b6-a06a971f84a3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:03.924946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fee2e2-c48d-4cd7-b4b6-a06a971f84a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:03.925225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fee2e2-c48d-4cd7-b4b6-a06a971f84a3 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:03.926030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fee2e2-c48d-4cd7-b4b6-a06a971f84a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:03.926343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fee2e2-c48d-4cd7-b4b6-a06a971f84a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:03.927743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fee2e2-c48d-4cd7-b4b6-a06a971f84a3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:03.935915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 25/50] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:03.937709 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f97eb-133b-4975-b872-62b5ee0cb6ab None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:03.938068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f97eb-133b-4975-b872-62b5ee0cb6ab None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:03.938424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f97eb-133b-4975-b872-62b5ee0cb6ab None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:03.940503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 26/51] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:39:03.941309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fee2e2-c48d-4cd7-b4b6-a06a971f84a3 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:03.945011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2c2e-f269-4b1f-9ca6-c7695708d1af None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:03.945303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2c2e-f269-4b1f-9ca6-c7695708d1af None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:03.945599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c2c2e-f269-4b1f-9ca6-c7695708d1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:03.961964 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-0f8c2c2e-f269-4b1f-9ca6-c7695708d1af None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:03.963178 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0f8c2c2e-f269-4b1f-9ca6-c7695708d1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:04.031290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 27/52] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:04.039864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fee2e2-c48d-4cd7-b4b6-a06a971f84a3 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:04.055782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c2c2e-f269-4b1f-9ca6-c7695708d1af None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:04.074150 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c5f87-0a77-4cf3-8f26-57a9a0013d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:04.075031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c5f87-0a77-4cf3-8f26-57a9a0013d81 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:04.075313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c5f87-0a77-4cf3-8f26-57a9a0013d81 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:04.075590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c5f87-0a77-4cf3-8f26-57a9a0013d81 None admin] PATH_INFO: `/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:04.076795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c5f87-0a77-4cf3-8f26-57a9a0013d81 None admin] RBAC: Authorizing `identity:get_user(user_id=9d09bdd9927e44ffbfd9b2ec447a5887)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:04.078293 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c5f87-0a77-4cf3-8f26-57a9a0013d81 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:04.080580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 26/53] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:03 2022] GET /identity/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:04.085613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:04.085810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c5f87-0a77-4cf3-8f26-57a9a0013d81 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:04.129796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:04.207920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c5f87-0a77-4cf3-8f26-57a9a0013d81 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:04.241887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3afda-ec94-43a0-838f-59dcc029b537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:04.242683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3afda-ec94-43a0-838f-59dcc029b537 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:04.242969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3afda-ec94-43a0-838f-59dcc029b537 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:04.243251 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3afda-ec94-43a0-838f-59dcc029b537 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:04.244439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3afda-ec94-43a0-838f-59dcc029b537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:04.245935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3afda-ec94-43a0-838f-59dcc029b537 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:04.247569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 27/54] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:04.253272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3afda-ec94-43a0-838f-59dcc029b537 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:04.269803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c2c2e-f269-4b1f-9ca6-c7695708d1af None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:04.304372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0623f-df10-44a7-a232-2ec3da2d3933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:04.305150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0623f-df10-44a7-a232-2ec3da2d3933 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:04.305434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0623f-df10-44a7-a232-2ec3da2d3933 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:04.305740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0623f-df10-44a7-a232-2ec3da2d3933 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:04.306922 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0623f-df10-44a7-a232-2ec3da2d3933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:04.308370 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0623f-df10-44a7-a232-2ec3da2d3933 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:04.310032 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 28/55] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:04.315650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0623f-df10-44a7-a232-2ec3da2d3933 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:04.354436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3afda-ec94-43a0-838f-59dcc029b537 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:04.363223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:04.387832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d768b8-a462-4aaa-9d57-3807bb176fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:04.388489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d768b8-a462-4aaa-9d57-3807bb176fe6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:04.388776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d768b8-a462-4aaa-9d57-3807bb176fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:04.389060 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d768b8-a462-4aaa-9d57-3807bb176fe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:04.389937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d768b8-a462-4aaa-9d57-3807bb176fe6 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.domain.id=default, user.id=9d09bdd9927e44ffbfd9b2ec447a5887) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:04.390260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d768b8-a462-4aaa-9d57-3807bb176fe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:04.391713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d768b8-a462-4aaa-9d57-3807bb176fe6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:04.398096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 28/56] 10.222.0.66 () {56 vars in 1322 bytes} [Fri Jul 1 19:39:04 2022] GET /identity/v3/role_assignments?user.id=9d09bdd9927e44ffbfd9b2ec447a5887&scope.domain.id=default&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:04.414537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:04.417499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0623f-df10-44a7-a232-2ec3da2d3933 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:04.455463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1ae85-ef40-4c25-97cd-e68f9c5ac79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:04.456154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1ae85-ef40-4c25-97cd-e68f9c5ac79f None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:04.456439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1ae85-ef40-4c25-97cd-e68f9c5ac79f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:04.456716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1ae85-ef40-4c25-97cd-e68f9c5ac79f None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:04.457458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1ae85-ef40-4c25-97cd-e68f9c5ac79f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:04.459014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1ae85-ef40-4c25-97cd-e68f9c5ac79f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:04.486718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 29/57] 10.222.0.66 () {60 vars in 1140 bytes} [Fri Jul 1 19:39:04 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:39:06.533618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdb907-c8b6-4e09-866d-eeed50da992c None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:06.534121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdb907-c8b6-4e09-866d-eeed50da992c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:06.534223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdb907-c8b6-4e09-866d-eeed50da992c None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:06.535570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 29/58] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:06.540976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e5be1-b7ed-41da-920a-e0c73ecf3cee None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:06.541357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e5be1-b7ed-41da-920a-e0c73ecf3cee None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:06.541537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e5be1-b7ed-41da-920a-e0c73ecf3cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:06.558678 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7c7e5be1-b7ed-41da-920a-e0c73ecf3cee None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:06.559882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7c7e5be1-b7ed-41da-920a-e0c73ecf3cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:06.624090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 30/59] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:06.649468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdb907-c8b6-4e09-866d-eeed50da992c None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:06.748419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdb907-c8b6-4e09-866d-eeed50da992c None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:06.780577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dda7ef-7fb2-457b-991c-5671c67d7c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:06.781363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dda7ef-7fb2-457b-991c-5671c67d7c7c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:06.781672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dda7ef-7fb2-457b-991c-5671c67d7c7c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:06.781950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dda7ef-7fb2-457b-991c-5671c67d7c7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:06.783708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dda7ef-7fb2-457b-991c-5671c67d7c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:06.785099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dda7ef-7fb2-457b-991c-5671c67d7c7c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:06.786994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 30/60] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:06.792332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e5be1-b7ed-41da-920a-e0c73ecf3cee None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:06.871769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:06.890283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e5be1-b7ed-41da-920a-e0c73ecf3cee None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:06.924895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539127b9-b718-4a0f-a00e-e0d7fcb6d161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:06.925559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539127b9-b718-4a0f-a00e-e0d7fcb6d161 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:06.925883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539127b9-b718-4a0f-a00e-e0d7fcb6d161 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:06.926162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539127b9-b718-4a0f-a00e-e0d7fcb6d161 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:06.926884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539127b9-b718-4a0f-a00e-e0d7fcb6d161 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:06.928333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539127b9-b718-4a0f-a00e-e0d7fcb6d161 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:06.952391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 31/61] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:39:06 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:39:10.123777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf1ec-bf2b-435f-ad67-5fa06e61349f None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.124812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf1ec-bf2b-435f-ad67-5fa06e61349f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.125295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf1ec-bf2b-435f-ad67-5fa06e61349f None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.126928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 31/62] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:10.132601 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8700be8-a7df-48f7-b00a-58466b7e4cda None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.133482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8700be8-a7df-48f7-b00a-58466b7e4cda None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.134016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8700be8-a7df-48f7-b00a-58466b7e4cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.153298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e8700be8-a7df-48f7-b00a-58466b7e4cda None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:10.154876 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e8700be8-a7df-48f7-b00a-58466b7e4cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:10.250647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 32/63] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:10.300351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbf1ec-bf2b-435f-ad67-5fa06e61349f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:10.311108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca18e5e-c6ba-48aa-8b5e-62a1717f5c1b None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.314440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca18e5e-c6ba-48aa-8b5e-62a1717f5c1b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.314940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca18e5e-c6ba-48aa-8b5e-62a1717f5c1b None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.317150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 33/64] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:10 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:10.322690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5159d36-7726-4bee-b93c-d19855921f35 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.323629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5159d36-7726-4bee-b93c-d19855921f35 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.323921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5159d36-7726-4bee-b93c-d19855921f35 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.325451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 34/65] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:10.326685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f8ee7-9b30-41ef-b93e-948ff9b11538 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.326975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f8ee7-9b30-41ef-b93e-948ff9b11538 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.327254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f8ee7-9b30-41ef-b93e-948ff9b11538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.345029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d39f8ee7-9b30-41ef-b93e-948ff9b11538 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:10.346360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d39f8ee7-9b30-41ef-b93e-948ff9b11538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:10.355840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:10.414956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:10.419022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbf1ec-bf2b-435f-ad67-5fa06e61349f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:10.425155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 35/66] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:10.427105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c47785-d2e6-4d29-9e38-d55f116d9b6e None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.427605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c47785-d2e6-4d29-9e38-d55f116d9b6e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.428074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c47785-d2e6-4d29-9e38-d55f116d9b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.458320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b69aa3-100d-4989-aa9f-47fee45fefb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:10.459502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b69aa3-100d-4989-aa9f-47fee45fefb7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.459616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-17c47785-d2e6-4d29-9e38-d55f116d9b6e None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:10.460022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b69aa3-100d-4989-aa9f-47fee45fefb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.460452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b69aa3-100d-4989-aa9f-47fee45fefb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.460984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-17c47785-d2e6-4d29-9e38-d55f116d9b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:10.474406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b69aa3-100d-4989-aa9f-47fee45fefb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-18b69aa3-100d-4989-aa9f-47fee45fefb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:10.484600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:10.485738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:10.486683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:10.489685 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 32/67] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:10.494826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b69aa3-100d-4989-aa9f-47fee45fefb7 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:10.546018 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 36/68] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:10.547283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c47785-d2e6-4d29-9e38-d55f116d9b6e None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:10.623489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:10.628110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b69aa3-100d-4989-aa9f-47fee45fefb7 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:10.658071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c47785-d2e6-4d29-9e38-d55f116d9b6e None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:10.673641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f99f9-6e08-4ce4-94e9-fbd2269970ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:10.674346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f99f9-6e08-4ce4-94e9-fbd2269970ee None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.674632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f99f9-6e08-4ce4-94e9-fbd2269970ee None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.675460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f99f9-6e08-4ce4-94e9-fbd2269970ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.688995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f99f9-6e08-4ce4-94e9-fbd2269970ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:10.690936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-bc4f99f9-6e08-4ce4-94e9-fbd2269970ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:10.698474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:10.699336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:10.700597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:10.701659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 33/69] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:10.702508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f99f9-6e08-4ce4-94e9-fbd2269970ee None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:10.704658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4d3bf-3490-4dd9-87e3-218a85b157a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:10.705326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4d3bf-3490-4dd9-87e3-218a85b157a1 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.705761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4d3bf-3490-4dd9-87e3-218a85b157a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.706003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4d3bf-3490-4dd9-87e3-218a85b157a1 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.707381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4d3bf-3490-4dd9-87e3-218a85b157a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:10.707693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4d3bf-3490-4dd9-87e3-218a85b157a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:10.709101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4d3bf-3490-4dd9-87e3-218a85b157a1 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:10.718092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 37/70] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:10.719316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16bbdc-b597-4b55-8efa-25de4e450665 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.719604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16bbdc-b597-4b55-8efa-25de4e450665 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.719878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16bbdc-b597-4b55-8efa-25de4e450665 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.721256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 38/71] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:10.722845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16bbdc-b597-4b55-8efa-25de4e450665 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:10.815805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f99f9-6e08-4ce4-94e9-fbd2269970ee None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:10.850704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16bbdc-b597-4b55-8efa-25de4e450665 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:10.876174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c24f6-4f2b-469a-a877-4c046e13fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:10.876994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c24f6-4f2b-469a-a877-4c046e13fb65 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.877274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c24f6-4f2b-469a-a877-4c046e13fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.877545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c24f6-4f2b-469a-a877-4c046e13fb65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.887435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffc289-a97e-4331-8e14-4fb1030758af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:10.888096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc289-a97e-4331-8e14-4fb1030758af None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.888386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc289-a97e-4331-8e14-4fb1030758af None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.888662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc289-a97e-4331-8e14-4fb1030758af None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.889406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffc289-a97e-4331-8e14-4fb1030758af None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:10.889740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffc289-a97e-4331-8e14-4fb1030758af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:10.889829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c24f6-4f2b-469a-a877-4c046e13fb65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:10.891136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffc289-a97e-4331-8e14-4fb1030758af None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-f83c24f6-4f2b-469a-a877-4c046e13fb65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:10.898528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:10.899426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:10.900175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:10.900729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 39/72] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:10.900793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 34/73] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:10.900862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffc289-a97e-4331-8e14-4fb1030758af None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:10.900934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e8aa-63d0-46f0-992f-e3598bf18145 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.901522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e8aa-63d0-46f0-992f-e3598bf18145 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.901522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749e8aa-63d0-46f0-992f-e3598bf18145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.919001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6749e8aa-63d0-46f0-992f-e3598bf18145 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:10.920190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6749e8aa-63d0-46f0-992f-e3598bf18145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:10.936541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:10.959485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:10.989617 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 35/74] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:10.990757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8b99e-125a-4b55-a94a-af0298c55b79 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:10.991052 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8b99e-125a-4b55-a94a-af0298c55b79 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:10.991342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8b99e-125a-4b55-a94a-af0298c55b79 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:10.992618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 36/75] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:10.996649 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8b99e-125a-4b55-a94a-af0298c55b79 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:11.011532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffc289-a97e-4331-8e14-4fb1030758af None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.047543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ed395-f661-4a50-a94f-140eb4a68124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.048410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ed395-f661-4a50-a94f-140eb4a68124 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.048693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ed395-f661-4a50-a94f-140eb4a68124 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.048970 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ed395-f661-4a50-a94f-140eb4a68124 None admin] PATH_INFO: `/v3/users/72cd343fe8fe49438f4bb8d66709e6ad` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.064149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ed395-f661-4a50-a94f-140eb4a68124 None admin] RBAC: Authorizing `identity:get_user(user_id=72cd343fe8fe49438f4bb8d66709e6ad)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:11.066206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ed395-f661-4a50-a94f-140eb4a68124 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:11.068044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 40/76] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:10 2022] GET /identity/v3/users/72cd343fe8fe49438f4bb8d66709e6ad => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:11.068666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ed395-f661-4a50-a94f-140eb4a68124 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:11.095720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8b99e-125a-4b55-a94a-af0298c55b79 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.130137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626ef72-47a4-402c-9600-e7c0d563177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.130807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626ef72-47a4-402c-9600-e7c0d563177d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.131090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626ef72-47a4-402c-9600-e7c0d563177d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.131367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626ef72-47a4-402c-9600-e7c0d563177d None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.132119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626ef72-47a4-402c-9600-e7c0d563177d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:11.132425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626ef72-47a4-402c-9600-e7c0d563177d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:11.133850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626ef72-47a4-402c-9600-e7c0d563177d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:11.146191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 37/77] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:11.146191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd541ed-fd4b-46ee-b208-6771a8736b28 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.146191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd541ed-fd4b-46ee-b208-6771a8736b28 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.146191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd541ed-fd4b-46ee-b208-6771a8736b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.164680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5bd541ed-fd4b-46ee-b208-6771a8736b28 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:11.165924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5bd541ed-fd4b-46ee-b208-6771a8736b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:11.173499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:11.191205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ed395-f661-4a50-a94f-140eb4a68124 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.201649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:11.226441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cc7c3-dfdc-4fb6-8a70-eaffc314eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.227316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cc7c3-dfdc-4fb6-8a70-eaffc314eea5 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.227602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cc7c3-dfdc-4fb6-8a70-eaffc314eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.227883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cc7c3-dfdc-4fb6-8a70-eaffc314eea5 None admin] PATH_INFO: `/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.229089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cc7c3-dfdc-4fb6-8a70-eaffc314eea5 None admin] RBAC: Authorizing `identity:get_user(user_id=9d09bdd9927e44ffbfd9b2ec447a5887)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:11.230598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cc7c3-dfdc-4fb6-8a70-eaffc314eea5 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:11.232237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 41/78] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:11 2022] GET /identity/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:11.249844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cc7c3-dfdc-4fb6-8a70-eaffc314eea5 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:11.250023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 38/79] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:11.250023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd541ed-fd4b-46ee-b208-6771a8736b28 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:11.334635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd541ed-fd4b-46ee-b208-6771a8736b28 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.343322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cc7c3-dfdc-4fb6-8a70-eaffc314eea5 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.368700 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5661f98-2af6-4e7c-8ced-ec24a0632eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.369354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5661f98-2af6-4e7c-8ced-ec24a0632eaf None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.369677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5661f98-2af6-4e7c-8ced-ec24a0632eaf None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.369954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5661f98-2af6-4e7c-8ced-ec24a0632eaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.378316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb549b-7e0a-47de-ab76-a5f6772956db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.379053 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb549b-7e0a-47de-ab76-a5f6772956db None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.379383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb549b-7e0a-47de-ab76-a5f6772956db None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.379627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb549b-7e0a-47de-ab76-a5f6772956db None admin] PATH_INFO: `/v3/roles/member` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.381787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:11.390667 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb549b-7e0a-47de-ab76-a5f6772956db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-8feb549b-7e0a-47de-ab76-a5f6772956db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:11.398664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:11.399508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:11.400394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 42/80] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:11.401553 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb549b-7e0a-47de-ab76-a5f6772956db None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:11.441083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:11.442081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5661f98-2af6-4e7c-8ced-ec24a0632eaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:11.443580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5661f98-2af6-4e7c-8ced-ec24a0632eaf None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:11.503656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb549b-7e0a-47de-ab76-a5f6772956db None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.537489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe91df-e7e4-46e9-a292-3a194865b5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.538399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe91df-e7e4-46e9-a292-3a194865b5d4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.538683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe91df-e7e4-46e9-a292-3a194865b5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.538956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe91df-e7e4-46e9-a292-3a194865b5d4 None admin] PATH_INFO: `/v3/users/72cd343fe8fe49438f4bb8d66709e6ad` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.540174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe91df-e7e4-46e9-a292-3a194865b5d4 None admin] RBAC: Authorizing `identity:get_user(user_id=72cd343fe8fe49438f4bb8d66709e6ad)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:11.541645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe91df-e7e4-46e9-a292-3a194865b5d4 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:11.543281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 43/82] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:11 2022] GET /identity/v3/users/72cd343fe8fe49438f4bb8d66709e6ad => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:11.548347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 39/82] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:11.548429 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe91df-e7e4-46e9-a292-3a194865b5d4 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:11.548499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5661f98-2af6-4e7c-8ced-ec24a0632eaf None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:11.607208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:11.643186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe91df-e7e4-46e9-a292-3a194865b5d4 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.652025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:11.666099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5661f98-2af6-4e7c-8ced-ec24a0632eaf None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.679526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caafd91-33c1-473d-97da-bc2e6ab6ce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.680168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caafd91-33c1-473d-97da-bc2e6ab6ce87 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.680475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caafd91-33c1-473d-97da-bc2e6ab6ce87 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.680761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caafd91-33c1-473d-97da-bc2e6ab6ce87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.700217 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e3a8b-839c-445b-baf7-55f6855147f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.700942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e3a8b-839c-445b-baf7-55f6855147f8 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.701224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e3a8b-839c-445b-baf7-55f6855147f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.701491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e3a8b-839c-445b-baf7-55f6855147f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.702690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e3a8b-839c-445b-baf7-55f6855147f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:11.704141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e3a8b-839c-445b-baf7-55f6855147f8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:11.705768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 40/83] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:11.707717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e3a8b-839c-445b-baf7-55f6855147f8 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:11.751397 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caafd91-33c1-473d-97da-bc2e6ab6ce87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:11.753007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caafd91-33c1-473d-97da-bc2e6ab6ce87 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:11.807400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e3a8b-839c-445b-baf7-55f6855147f8 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.818086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:11.842287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39aecb2-1cfb-464a-b14d-b863cc11e585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.842951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39aecb2-1cfb-464a-b14d-b863cc11e585 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.843235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39aecb2-1cfb-464a-b14d-b863cc11e585 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.843514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39aecb2-1cfb-464a-b14d-b863cc11e585 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.844264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39aecb2-1cfb-464a-b14d-b863cc11e585 None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:11.844571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39aecb2-1cfb-464a-b14d-b863cc11e585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:11.845986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39aecb2-1cfb-464a-b14d-b863cc11e585 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:11.849652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 44/84] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:11.850492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caafd91-33c1-473d-97da-bc2e6ab6ce87 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:11.857615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 41/85] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:11.858422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39aecb2-1cfb-464a-b14d-b863cc11e585 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:11.860355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:11.955896 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caafd91-33c1-473d-97da-bc2e6ab6ce87 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.963173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39aecb2-1cfb-464a-b14d-b863cc11e585 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:11.989504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89fdbc-7baa-4d33-81d2-072b2d1f5d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.990174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89fdbc-7baa-4d33-81d2-072b2d1f5d98 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.990462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89fdbc-7baa-4d33-81d2-072b2d1f5d98 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.990737 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89fdbc-7baa-4d33-81d2-072b2d1f5d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.995536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b7641-bb3f-4ea7-97e4-a97f23aa18ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:11.996396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b7641-bb3f-4ea7-97e4-a97f23aa18ef None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:11.996683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b7641-bb3f-4ea7-97e4-a97f23aa18ef None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:11.996962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b7641-bb3f-4ea7-97e4-a97f23aa18ef None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:11.998170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b7641-bb3f-4ea7-97e4-a97f23aa18ef None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:11.999626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b7641-bb3f-4ea7-97e4-a97f23aa18ef None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:12.001489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 42/86] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:11 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:12.002186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b7641-bb3f-4ea7-97e4-a97f23aa18ef None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:12.045939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:12.059596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89fdbc-7baa-4d33-81d2-072b2d1f5d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:12.061054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89fdbc-7baa-4d33-81d2-072b2d1f5d98 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:12.069252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:12.104360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b7641-bb3f-4ea7-97e4-a97f23aa18ef None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:12.138114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9066f9d-f836-40d3-ab1f-c1d5a8585117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:12.138785 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066f9d-f836-40d3-ab1f-c1d5a8585117 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:12.139064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066f9d-f836-40d3-ab1f-c1d5a8585117 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:12.139348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066f9d-f836-40d3-ab1f-c1d5a8585117 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:12.140297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9066f9d-f836-40d3-ab1f-c1d5a8585117 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:12.141561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9066f9d-f836-40d3-ab1f-c1d5a8585117 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:12.158736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 45/87] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:12.160204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89fdbc-7baa-4d33-81d2-072b2d1f5d98 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:12.169046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 43/88] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:39:12 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:12.169898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9066f9d-f836-40d3-ab1f-c1d5a8585117 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:12.276161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:12.283420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9066f9d-f836-40d3-ab1f-c1d5a8585117 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:12.291245 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89fdbc-7baa-4d33-81d2-072b2d1f5d98 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:12.302701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:12.328245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c245e-6d01-4a0a-a797-1326dd2b46f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:12.329217 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c245e-6d01-4a0a-a797-1326dd2b46f6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:12.329505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c245e-6d01-4a0a-a797-1326dd2b46f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:12.329817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c245e-6d01-4a0a-a797-1326dd2b46f6 None admin] PATH_INFO: `/v3/users/72cd343fe8fe49438f4bb8d66709e6ad` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:12.332641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c245e-6d01-4a0a-a797-1326dd2b46f6 None admin] RBAC: Authorizing `identity:get_user(user_id=72cd343fe8fe49438f4bb8d66709e6ad)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:12.332747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbe457-f6b6-4383-97c7-5310222d784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:12.333195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbe457-f6b6-4383-97c7-5310222d784c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:12.334928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbe457-f6b6-4383-97c7-5310222d784c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:12.335045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c245e-6d01-4a0a-a797-1326dd2b46f6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:12.335120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccbe457-f6b6-4383-97c7-5310222d784c None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:12.335120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbe457-f6b6-4383-97c7-5310222d784c None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:12.335411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 44/89] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:12 2022] GET /identity/v3/users/72cd343fe8fe49438f4bb8d66709e6ad => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:12.336674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c245e-6d01-4a0a-a797-1326dd2b46f6 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:12.338786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccbe457-f6b6-4383-97c7-5310222d784c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:12.340587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 46/90] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:12 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:12.341371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbe457-f6b6-4383-97c7-5310222d784c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:12.462128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c245e-6d01-4a0a-a797-1326dd2b46f6 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:12.468442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccbe457-f6b6-4383-97c7-5310222d784c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:12.504853 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bedd5e-77be-4659-b8b7-dc329ce19a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:12.505903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bedd5e-77be-4659-b8b7-dc329ce19a03 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:12.506361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bedd5e-77be-4659-b8b7-dc329ce19a03 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:12.507381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bedd5e-77be-4659-b8b7-dc329ce19a03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:12.507484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bedd5e-77be-4659-b8b7-dc329ce19a03 None admin] RBAC: Adding query filter params (role.id=2e1db0cf27684f389b9320673721684d, scope.project.id=574afa1590854c65a807df4589e2c460, user.id=72cd343fe8fe49438f4bb8d66709e6ad) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:12.507788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bedd5e-77be-4659-b8b7-dc329ce19a03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:12.509244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bedd5e-77be-4659-b8b7-dc329ce19a03 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:12.511787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ff177-c47b-44ea-933c-a9cd714f5096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:12.512863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ff177-c47b-44ea-933c-a9cd714f5096 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:12.513344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ff177-c47b-44ea-933c-a9cd714f5096 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:12.513820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ff177-c47b-44ea-933c-a9cd714f5096 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:12.515184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ff177-c47b-44ea-933c-a9cd714f5096 None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:12.515642 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 45/91] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:12 2022] GET /identity/v3/role_assignments?user.id=72cd343fe8fe49438f4bb8d66709e6ad&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=2e1db0cf27684f389b9320673721684d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:12.516363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bedd5e-77be-4659-b8b7-dc329ce19a03 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:12.517407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ff177-c47b-44ea-933c-a9cd714f5096 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:12.519383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 47/92] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:12 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:12.520312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ff177-c47b-44ea-933c-a9cd714f5096 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:12.589878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:12.595463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:12.716240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bedd5e-77be-4659-b8b7-dc329ce19a03 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:12.716995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ff177-c47b-44ea-933c-a9cd714f5096 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:12.770079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b804-560b-493e-ad35-a4a8341b2441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:12.770750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b804-560b-493e-ad35-a4a8341b2441 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:12.771033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b804-560b-493e-ad35-a4a8341b2441 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:12.771309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3b804-560b-493e-ad35-a4a8341b2441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:12.772551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9448e5-4a19-44ce-bfc5-a28fd9e7afcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:12.773207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9448e5-4a19-44ce-bfc5-a28fd9e7afcc None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:12.773481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9448e5-4a19-44ce-bfc5-a28fd9e7afcc None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:12.773788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9448e5-4a19-44ce-bfc5-a28fd9e7afcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:12.775273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9448e5-4a19-44ce-bfc5-a28fd9e7afcc None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=574afa1590854c65a807df4589e2c460, user.id=9d09bdd9927e44ffbfd9b2ec447a5887) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:12.775582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9448e5-4a19-44ce-bfc5-a28fd9e7afcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:12.777044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9448e5-4a19-44ce-bfc5-a28fd9e7afcc None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:12.789607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 48/93] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:12 2022] GET /identity/v3/role_assignments?user.id=9d09bdd9927e44ffbfd9b2ec447a5887&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:12.789607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9448e5-4a19-44ce-bfc5-a28fd9e7afcc None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:12.942189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3b804-560b-493e-ad35-a4a8341b2441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:12.943601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3b804-560b-493e-ad35-a4a8341b2441 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:12.984935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9448e5-4a19-44ce-bfc5-a28fd9e7afcc None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:13.001270 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:13.030750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:13.059734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9e443-ba73-4328-8ace-5fb8e0ae9921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:13.060405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e443-ba73-4328-8ace-5fb8e0ae9921 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:13.060686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e443-ba73-4328-8ace-5fb8e0ae9921 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:13.060960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e443-ba73-4328-8ace-5fb8e0ae9921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:13.061842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9e443-ba73-4328-8ace-5fb8e0ae9921 None admin] RBAC: Adding query filter params (role.id=b127031d393048d7ae9a35015113e82a, scope.project.id=574afa1590854c65a807df4589e2c460, user.id=72cd343fe8fe49438f4bb8d66709e6ad) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:13.062152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9e443-ba73-4328-8ace-5fb8e0ae9921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:13.063600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9e443-ba73-4328-8ace-5fb8e0ae9921 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:13.081767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 49/94] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:12 2022] GET /identity/v3/role_assignments?user.id=72cd343fe8fe49438f4bb8d66709e6ad&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=b127031d393048d7ae9a35015113e82a => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:13.158039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 46/95] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:13.164526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9e443-ba73-4328-8ace-5fb8e0ae9921 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:13.303024 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9e443-ba73-4328-8ace-5fb8e0ae9921 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:13.362882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fed7b-7067-43a1-80bb-366f149c1df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:13.363764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fed7b-7067-43a1-80bb-366f149c1df2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:13.364048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fed7b-7067-43a1-80bb-366f149c1df2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:13.364322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fed7b-7067-43a1-80bb-366f149c1df2 None admin] PATH_INFO: `/v3/projects/4debe830e03247c79d3889e7479b9dfa` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:13.365557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fed7b-7067-43a1-80bb-366f149c1df2 None admin] RBAC: Authorizing `identity:get_project(project_id=4debe830e03247c79d3889e7479b9dfa)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:13.367050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fed7b-7067-43a1-80bb-366f149c1df2 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:13.368991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 50/96] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:13 2022] GET /identity/v3/projects/4debe830e03247c79d3889e7479b9dfa => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:13.375109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b804-560b-493e-ad35-a4a8341b2441 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:13.488795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3b804-560b-493e-ad35-a4a8341b2441 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:13.516197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:13.525135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb091637-626f-4143-bed4-663c887b3bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:13.525837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb091637-626f-4143-bed4-663c887b3bbc None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:13.526134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb091637-626f-4143-bed4-663c887b3bbc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:13.526398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb091637-626f-4143-bed4-663c887b3bbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:13.527299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb091637-626f-4143-bed4-663c887b3bbc None admin] RBAC: Adding query filter params (role.id=2e1db0cf27684f389b9320673721684d, scope.project.id=4debe830e03247c79d3889e7479b9dfa, user.id=72cd343fe8fe49438f4bb8d66709e6ad) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:13.527614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb091637-626f-4143-bed4-663c887b3bbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:13.529337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb091637-626f-4143-bed4-663c887b3bbc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:13.541804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 47/97] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:13 2022] GET /identity/v3/role_assignments?user.id=72cd343fe8fe49438f4bb8d66709e6ad&scope.project.id=4debe830e03247c79d3889e7479b9dfa&role.id=2e1db0cf27684f389b9320673721684d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:14.973242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f309274-eee9-4b32-b675-65f4db4cf441 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:14.973780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f309274-eee9-4b32-b675-65f4db4cf441 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:14.973854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f309274-eee9-4b32-b675-65f4db4cf441 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:14.977368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 51/98] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:14.983413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ab4c8-e73f-4225-8d93-691e608c5c7f None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:14.983413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ab4c8-e73f-4225-8d93-691e608c5c7f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:14.983413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ab4c8-e73f-4225-8d93-691e608c5c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:15.003790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ba8ab4c8-e73f-4225-8d93-691e608c5c7f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:15.004594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ba8ab4c8-e73f-4225-8d93-691e608c5c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:15.093471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 48/99] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:15.151306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f309274-eee9-4b32-b675-65f4db4cf441 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:15.199661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:15.270987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f309274-eee9-4b32-b675-65f4db4cf441 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:15.284465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729d4eb-b600-4ff4-888b-5920ebe9deec None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:15.285234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729d4eb-b600-4ff4-888b-5920ebe9deec None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:15.285525 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729d4eb-b600-4ff4-888b-5920ebe9deec None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:15.287205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 49/100] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:39:15.292637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b10661-2e6b-44ef-896e-3c3b5d1639a1 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:15.292929 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b10661-2e6b-44ef-896e-3c3b5d1639a1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:15.293210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b10661-2e6b-44ef-896e-3c3b5d1639a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:15.318536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f9b10661-2e6b-44ef-896e-3c3b5d1639a1 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:15.319753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f9b10661-2e6b-44ef-896e-3c3b5d1639a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:15.350712 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bac417-2ce1-40ed-837b-c22e4f17fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:15.351520 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bac417-2ce1-40ed-837b-c22e4f17fc6d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:15.351805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bac417-2ce1-40ed-837b-c22e4f17fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:15.352079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bac417-2ce1-40ed-837b-c22e4f17fc6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:15.371502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bac417-2ce1-40ed-837b-c22e4f17fc6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-a5bac417-2ce1-40ed-837b-c22e4f17fc6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:15.383478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:15.384335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:15.385064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:15.387087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 52/101] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:15.395849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bac417-2ce1-40ed-837b-c22e4f17fc6d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:15.462914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 50/102] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:15.525222 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b51a4-9d77-4840-9369-c409d5f9b859 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:15.525506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b51a4-9d77-4840-9369-c409d5f9b859 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:15.525742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b51a4-9d77-4840-9369-c409d5f9b859 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:15.527285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 51/103] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:39:15.533222 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653a53b-5a2e-4e9b-b87c-01918d8c3bd2 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:15.533514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653a53b-5a2e-4e9b-b87c-01918d8c3bd2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:15.533820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653a53b-5a2e-4e9b-b87c-01918d8c3bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:15.574646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a653a53b-5a2e-4e9b-b87c-01918d8c3bd2 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:15.575873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a653a53b-5a2e-4e9b-b87c-01918d8c3bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:15.576448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bac417-2ce1-40ed-837b-c22e4f17fc6d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:15.590524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:15.640727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b090a5-9940-49b5-bbf1-ef7d4e0c5d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:15.641402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b090a5-9940-49b5-bbf1-ef7d4e0c5d2d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:15.641710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b090a5-9940-49b5-bbf1-ef7d4e0c5d2d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:15.642038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b090a5-9940-49b5-bbf1-ef7d4e0c5d2d None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:15.642791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b090a5-9940-49b5-bbf1-ef7d4e0c5d2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:15.643099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b090a5-9940-49b5-bbf1-ef7d4e0c5d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:15.645068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b090a5-9940-49b5-bbf1-ef7d4e0c5d2d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:15.657127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:15.658572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 53/104] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:15.661095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b090a5-9940-49b5-bbf1-ef7d4e0c5d2d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:15.694270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 52/105] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:15.695140 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653a53b-5a2e-4e9b-b87c-01918d8c3bd2 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:15.835978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653a53b-5a2e-4e9b-b87c-01918d8c3bd2 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:15.867000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b090a5-9940-49b5-bbf1-ef7d4e0c5d2d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:15.911156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3c99f-110e-46a2-a704-8a43a460585e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:15.912323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3c99f-110e-46a2-a704-8a43a460585e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:15.912609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3c99f-110e-46a2-a704-8a43a460585e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:15.912885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3c99f-110e-46a2-a704-8a43a460585e None admin] PATH_INFO: `/v3/users/c26dec6c07a148479302cc9f2e861a24` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:15.931772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adeede-4273-4e04-80cb-84a0113688ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:15.932969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adeede-4273-4e04-80cb-84a0113688ab None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:15.933450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adeede-4273-4e04-80cb-84a0113688ab None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:15.933943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adeede-4273-4e04-80cb-84a0113688ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:15.935660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adeede-4273-4e04-80cb-84a0113688ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:15.937545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adeede-4273-4e04-80cb-84a0113688ab None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:15.939803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 54/106] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:15.941196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adeede-4273-4e04-80cb-84a0113688ab None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:15.945714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3c99f-110e-46a2-a704-8a43a460585e None admin] RBAC: Authorizing `identity:get_user(user_id=c26dec6c07a148479302cc9f2e861a24)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:15.947430 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3c99f-110e-46a2-a704-8a43a460585e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:15.953449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 53/107] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:15 2022] GET /identity/v3/users/c26dec6c07a148479302cc9f2e861a24 => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:15.954489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3c99f-110e-46a2-a704-8a43a460585e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:15.974301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:16.057959 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:16.082362 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adeede-4273-4e04-80cb-84a0113688ab None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:16.122734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595b56f-d6f2-4f71-914e-6c114959b165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:16.124681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595b56f-d6f2-4f71-914e-6c114959b165 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.125174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595b56f-d6f2-4f71-914e-6c114959b165 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.125676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595b56f-d6f2-4f71-914e-6c114959b165 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.139711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595b56f-d6f2-4f71-914e-6c114959b165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-7595b56f-d6f2-4f71-914e-6c114959b165 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:16.151440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:16.152325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:16.153263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 55/108] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:16.154033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595b56f-d6f2-4f71-914e-6c114959b165 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:16.158339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3c99f-110e-46a2-a704-8a43a460585e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:16.193872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661cb16-1020-4add-aca6-3e8e8b5be173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:16.194951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661cb16-1020-4add-aca6-3e8e8b5be173 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.195441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661cb16-1020-4add-aca6-3e8e8b5be173 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.195913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661cb16-1020-4add-aca6-3e8e8b5be173 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.196823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661cb16-1020-4add-aca6-3e8e8b5be173 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:16.198825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661cb16-1020-4add-aca6-3e8e8b5be173 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:16.227986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 54/109] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:39:15 2022] POST /identity/v3/projects => generated 302 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:16.230449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d82b77-ace1-40ab-a199-7662a3071c1f None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.230745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d82b77-ace1-40ab-a199-7662a3071c1f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.231026 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d82b77-ace1-40ab-a199-7662a3071c1f None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.237606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 55/110] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:16.237606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d82b77-ace1-40ab-a199-7662a3071c1f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:16.291685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:16.319946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595b56f-d6f2-4f71-914e-6c114959b165 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:16.405801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9796ebc-9b53-4e9f-8131-87245c36c291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:16.406387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9796ebc-9b53-4e9f-8131-87245c36c291 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.407506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9796ebc-9b53-4e9f-8131-87245c36c291 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.407827 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9796ebc-9b53-4e9f-8131-87245c36c291 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.438464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:16.455595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d82b77-ace1-40ab-a199-7662a3071c1f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:16.497123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5111b1c-a186-4e14-90d8-c4839b9fb0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:16.498004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5111b1c-a186-4e14-90d8-c4839b9fb0a3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.498290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5111b1c-a186-4e14-90d8-c4839b9fb0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.498572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5111b1c-a186-4e14-90d8-c4839b9fb0a3 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.501958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5111b1c-a186-4e14-90d8-c4839b9fb0a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:16.502498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5111b1c-a186-4e14-90d8-c4839b9fb0a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:16.504656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5111b1c-a186-4e14-90d8-c4839b9fb0a3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:16.515578 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 56/111] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:16.517283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d356b0-61c9-4f45-809c-750114cb79d7 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.517874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d356b0-61c9-4f45-809c-750114cb79d7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.519102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d356b0-61c9-4f45-809c-750114cb79d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.519702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9796ebc-9b53-4e9f-8131-87245c36c291 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:16.524914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9796ebc-9b53-4e9f-8131-87245c36c291 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:16.538689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-92d356b0-61c9-4f45-809c-750114cb79d7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:16.540189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-92d356b0-61c9-4f45-809c-750114cb79d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:16.611189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 57/112] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:16.613244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a20b96-c9e5-46f0-81a0-aefd3188bd5f None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.613538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a20b96-c9e5-46f0-81a0-aefd3188bd5f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.613858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a20b96-c9e5-46f0-81a0-aefd3188bd5f None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.615272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 58/113] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:16.619732 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a20b96-c9e5-46f0-81a0-aefd3188bd5f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:16.641175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:16.656556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 56/114] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:16.658184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d60aa-2d36-4668-bcdb-585574187246 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.658645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d60aa-2d36-4668-bcdb-585574187246 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.659207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d60aa-2d36-4668-bcdb-585574187246 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.665099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 57/115] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:16.666462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799c150-f88c-41d4-8784-681bc0ae8789 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.666750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799c150-f88c-41d4-8784-681bc0ae8789 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.667025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799c150-f88c-41d4-8784-681bc0ae8789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.683204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-2799c150-f88c-41d4-8784-681bc0ae8789 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:16.684384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2799c150-f88c-41d4-8784-681bc0ae8789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:16.721113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:16.742014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a20b96-c9e5-46f0-81a0-aefd3188bd5f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:16.755940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 58/116] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:16.756801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799c150-f88c-41d4-8784-681bc0ae8789 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:16.790303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfe303-83e3-4669-80c1-5c49120dca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:16.791195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfe303-83e3-4669-80c1-5c49120dca55 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.791479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfe303-83e3-4669-80c1-5c49120dca55 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.791757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfe303-83e3-4669-80c1-5c49120dca55 None admin] PATH_INFO: `/v3/users/72cd343fe8fe49438f4bb8d66709e6ad` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.792955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfe303-83e3-4669-80c1-5c49120dca55 None admin] RBAC: Authorizing `identity:get_user(user_id=72cd343fe8fe49438f4bb8d66709e6ad)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:16.794470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfe303-83e3-4669-80c1-5c49120dca55 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:16.796107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 59/117] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:16 2022] GET /identity/v3/users/72cd343fe8fe49438f4bb8d66709e6ad => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:16.796881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfe303-83e3-4669-80c1-5c49120dca55 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:16.865699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799c150-f88c-41d4-8784-681bc0ae8789 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:16.884395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:16.904142 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cd676-75d8-43da-a2fe-50e9b64588fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:16.904943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd676-75d8-43da-a2fe-50e9b64588fa None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.905226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd676-75d8-43da-a2fe-50e9b64588fa None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.905504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd676-75d8-43da-a2fe-50e9b64588fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.906104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfe303-83e3-4669-80c1-5c49120dca55 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:16.917155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cd676-75d8-43da-a2fe-50e9b64588fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-ea2cd676-75d8-43da-a2fe-50e9b64588fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:16.925261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:16.926261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:16.927095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 59/118] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:16.927996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685db292-2cac-4028-915b-fe75f9c7c9f7 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.927996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685db292-2cac-4028-915b-fe75f9c7c9f7 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.928297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685db292-2cac-4028-915b-fe75f9c7c9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.945986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-685db292-2cac-4028-915b-fe75f9c7c9f7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:16.947190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-685db292-2cac-4028-915b-fe75f9c7c9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:16.950540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd84248-cd2a-432e-9fc3-a7b57b0702c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:16.951492 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd84248-cd2a-432e-9fc3-a7b57b0702c3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:16.951857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd84248-cd2a-432e-9fc3-a7b57b0702c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:16.952282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd84248-cd2a-432e-9fc3-a7b57b0702c3 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:16.953497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd84248-cd2a-432e-9fc3-a7b57b0702c3 None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:16.954974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd84248-cd2a-432e-9fc3-a7b57b0702c3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:16.962183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 60/119] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:16 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:16.962822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd84248-cd2a-432e-9fc3-a7b57b0702c3 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:16.985270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:17.020519 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 60/120] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:17.021518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685db292-2cac-4028-915b-fe75f9c7c9f7 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:17.076772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd84248-cd2a-432e-9fc3-a7b57b0702c3 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:17.115829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b72482-1e53-44fd-8395-88c5f127eea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:17.116669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b72482-1e53-44fd-8395-88c5f127eea1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:17.116955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b72482-1e53-44fd-8395-88c5f127eea1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:17.117236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b72482-1e53-44fd-8395-88c5f127eea1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:17.127558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b72482-1e53-44fd-8395-88c5f127eea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-e7b72482-1e53-44fd-8395-88c5f127eea1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:17.135469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:17.136280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 61/121] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:17.142508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b72482-1e53-44fd-8395-88c5f127eea1 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:17.143691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685db292-2cac-4028-915b-fe75f9c7c9f7 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:17.153112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:17.180461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f859a-4681-4ad4-8401-f37975d201c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:17.181379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f859a-4681-4ad4-8401-f37975d201c2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:17.181793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f859a-4681-4ad4-8401-f37975d201c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:17.182181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f859a-4681-4ad4-8401-f37975d201c2 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:17.183462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f859a-4681-4ad4-8401-f37975d201c2 None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:17.185006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f859a-4681-4ad4-8401-f37975d201c2 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:17.187036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 61/122] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:17 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:17.193040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f859a-4681-4ad4-8401-f37975d201c2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:17.228377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:17.250210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b72482-1e53-44fd-8395-88c5f127eea1 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:17.286227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b155c14-e583-449c-a00c-1199be377a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:17.287291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b155c14-e583-449c-a00c-1199be377a31 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:17.287568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b155c14-e583-449c-a00c-1199be377a31 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:17.287843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b155c14-e583-449c-a00c-1199be377a31 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:17.288593 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b155c14-e583-449c-a00c-1199be377a31 None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:17.288896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b155c14-e583-449c-a00c-1199be377a31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:17.290305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b155c14-e583-449c-a00c-1199be377a31 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:17.298805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 62/123] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:17.299489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f859a-4681-4ad4-8401-f37975d201c2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:17.304665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b155c14-e583-449c-a00c-1199be377a31 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:17.337054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0802ee9-03e6-48f0-b72e-2f6ac4346a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:17.337740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0802ee9-03e6-48f0-b72e-2f6ac4346a43 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:17.338026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0802ee9-03e6-48f0-b72e-2f6ac4346a43 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:17.338330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0802ee9-03e6-48f0-b72e-2f6ac4346a43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:17.339160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0802ee9-03e6-48f0-b72e-2f6ac4346a43 None admin] RBAC: Adding query filter params (role.id=0c5069e08baa4820b7154564a801ba7e, scope.project.id=574afa1590854c65a807df4589e2c460, user.id=c26dec6c07a148479302cc9f2e861a24) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:17.339475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0802ee9-03e6-48f0-b72e-2f6ac4346a43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:17.340919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0802ee9-03e6-48f0-b72e-2f6ac4346a43 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:17.346665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 62/124] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:17 2022] GET /identity/v3/role_assignments?user.id=c26dec6c07a148479302cc9f2e861a24&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=0c5069e08baa4820b7154564a801ba7e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:17.347459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0802ee9-03e6-48f0-b72e-2f6ac4346a43 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:17.402506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:17.438541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b155c14-e583-449c-a00c-1199be377a31 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:17.482068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5460c-16f5-40e5-abae-61503b77dcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:17.482956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5460c-16f5-40e5-abae-61503b77dcd0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:17.483186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5460c-16f5-40e5-abae-61503b77dcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:17.483483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5460c-16f5-40e5-abae-61503b77dcd0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:17.486209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0802ee9-03e6-48f0-b72e-2f6ac4346a43 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:17.496846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5460c-16f5-40e5-abae-61503b77dcd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-fde5460c-16f5-40e5-abae-61503b77dcd0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:17.505355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:17.506276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:17.507037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 63/125] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:17.507668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5460c-16f5-40e5-abae-61503b77dcd0 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:17.511795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:17.540529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31008d82-0ec6-427b-90e1-b959d203ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:17.541571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31008d82-0ec6-427b-90e1-b959d203ee20 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:17.541948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31008d82-0ec6-427b-90e1-b959d203ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:17.542294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31008d82-0ec6-427b-90e1-b959d203ee20 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:17.543388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31008d82-0ec6-427b-90e1-b959d203ee20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:17.544106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31008d82-0ec6-427b-90e1-b959d203ee20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:17.546134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31008d82-0ec6-427b-90e1-b959d203ee20 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:17.565702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 63/126] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:17.565702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31008d82-0ec6-427b-90e1-b959d203ee20 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:17.626826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5460c-16f5-40e5-abae-61503b77dcd0 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:17.664383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f41ec9-68db-456e-8b74-a749cf8f5a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:17.665386 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f41ec9-68db-456e-8b74-a749cf8f5a00 None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:17.665765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f41ec9-68db-456e-8b74-a749cf8f5a00 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:17.666115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f41ec9-68db-456e-8b74-a749cf8f5a00 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460/users/72cd343fe8fe49438f4bb8d66709e6ad/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:17.668429 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f41ec9-68db-456e-8b74-a749cf8f5a00 None admin] RBAC: Authorizing `identity:create_grant(project_id=574afa1590854c65a807df4589e2c460, user_id=72cd343fe8fe49438f4bb8d66709e6ad, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:17.670018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f41ec9-68db-456e-8b74-a749cf8f5a00 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:17.678041 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31008d82-0ec6-427b-90e1-b959d203ee20 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:17.679768 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:17.688128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 64/127] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:39:17 2022] PUT /identity/v3/projects/574afa1590854c65a807df4589e2c460/users/72cd343fe8fe49438f4bb8d66709e6ad/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:17.690079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f41ec9-68db-456e-8b74-a749cf8f5a00 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:17.761557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f2e2c-b9d5-4cce-979c-77d9a6977c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:17.762570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f2e2c-b9d5-4cce-979c-77d9a6977c2e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:17.762856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f2e2c-b9d5-4cce-979c-77d9a6977c2e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:17.763132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f2e2c-b9d5-4cce-979c-77d9a6977c2e None admin] PATH_INFO: `/v3/users/72cd343fe8fe49438f4bb8d66709e6ad` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:17.766135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:17.767186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f2e2c-b9d5-4cce-979c-77d9a6977c2e None admin] RBAC: Authorizing `identity:get_user(user_id=72cd343fe8fe49438f4bb8d66709e6ad)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:17.768917 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f2e2c-b9d5-4cce-979c-77d9a6977c2e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:17.778249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 64/128] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:17 2022] GET /identity/v3/users/72cd343fe8fe49438f4bb8d66709e6ad => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:17.779191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f2e2c-b9d5-4cce-979c-77d9a6977c2e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:17.811056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f41ec9-68db-456e-8b74-a749cf8f5a00 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:17.851938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0a12a-beb2-4c43-a1c6-0957ee9d8725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:17.853029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0a12a-beb2-4c43-a1c6-0957ee9d8725 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:17.853534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0a12a-beb2-4c43-a1c6-0957ee9d8725 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:17.854041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0a12a-beb2-4c43-a1c6-0957ee9d8725 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:17.855014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0a12a-beb2-4c43-a1c6-0957ee9d8725 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:17.855551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0a12a-beb2-4c43-a1c6-0957ee9d8725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:17.857182 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0a12a-beb2-4c43-a1c6-0957ee9d8725 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:17.866599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 65/129] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:17.867780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0a12a-beb2-4c43-a1c6-0957ee9d8725 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:17.980525 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0a12a-beb2-4c43-a1c6-0957ee9d8725 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:18.013382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:18.017286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f2e2c-b9d5-4cce-979c-77d9a6977c2e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:18.026141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1359e30-1280-422b-b2d5-3260535eb85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:18.027021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1359e30-1280-422b-b2d5-3260535eb85d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:18.027311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1359e30-1280-422b-b2d5-3260535eb85d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:18.028042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1359e30-1280-422b-b2d5-3260535eb85d None admin] PATH_INFO: `/v3/projects/4debe830e03247c79d3889e7479b9dfa` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:18.028958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1359e30-1280-422b-b2d5-3260535eb85d None admin] RBAC: Authorizing `identity:get_project(project_id=4debe830e03247c79d3889e7479b9dfa)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:18.030462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1359e30-1280-422b-b2d5-3260535eb85d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:18.032354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 66/130] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:17 2022] GET /identity/v3/projects/4debe830e03247c79d3889e7479b9dfa => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:18.032960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1359e30-1280-422b-b2d5-3260535eb85d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:18.071892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99430fa-0012-42e6-b356-cb9c0f434dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:18.073474 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99430fa-0012-42e6-b356-cb9c0f434dda None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:18.074240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99430fa-0012-42e6-b356-cb9c0f434dda None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:18.075604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99430fa-0012-42e6-b356-cb9c0f434dda None admin] PATH_INFO: `/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:18.077127 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99430fa-0012-42e6-b356-cb9c0f434dda None admin] RBAC: Authorizing `identity:get_user(user_id=9d09bdd9927e44ffbfd9b2ec447a5887)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:18.078847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99430fa-0012-42e6-b356-cb9c0f434dda None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:18.081618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 65/131] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:17 2022] GET /identity/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887 => generated 247 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:18.083074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99430fa-0012-42e6-b356-cb9c0f434dda None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:18.146689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:18.161251 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1359e30-1280-422b-b2d5-3260535eb85d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:18.203969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8d23c-6906-4d99-8215-7eec61959023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:18.204852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8d23c-6906-4d99-8215-7eec61959023 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:18.205135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8d23c-6906-4d99-8215-7eec61959023 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:18.205411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8d23c-6906-4d99-8215-7eec61959023 None admin] PATH_INFO: `/v3/users/72cd343fe8fe49438f4bb8d66709e6ad` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:18.206954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8d23c-6906-4d99-8215-7eec61959023 None admin] RBAC: Authorizing `identity:get_user(user_id=72cd343fe8fe49438f4bb8d66709e6ad)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:18.208526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8d23c-6906-4d99-8215-7eec61959023 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:18.213768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 67/132] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:18 2022] GET /identity/v3/users/72cd343fe8fe49438f4bb8d66709e6ad => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:18.214891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8d23c-6906-4d99-8215-7eec61959023 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:18.278201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:18.327843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8d23c-6906-4d99-8215-7eec61959023 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:18.339768 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99430fa-0012-42e6-b356-cb9c0f434dda None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:18.364495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521a40b-34b7-47ba-930b-bd46200db1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:18.365426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521a40b-34b7-47ba-930b-bd46200db1ac None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:18.365745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521a40b-34b7-47ba-930b-bd46200db1ac None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:18.366030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521a40b-34b7-47ba-930b-bd46200db1ac None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:18.367289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521a40b-34b7-47ba-930b-bd46200db1ac None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:18.368790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521a40b-34b7-47ba-930b-bd46200db1ac None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:18.370543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 68/133] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:18 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:18.375940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521a40b-34b7-47ba-930b-bd46200db1ac None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:18.438404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8fdd5-c4a1-44e1-845f-8586be356471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:18.439791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8fdd5-c4a1-44e1-845f-8586be356471 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:18.440360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8fdd5-c4a1-44e1-845f-8586be356471 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:18.441206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8fdd5-c4a1-44e1-845f-8586be356471 None admin] PATH_INFO: `/v3/projects/4debe830e03247c79d3889e7479b9dfa/users/72cd343fe8fe49438f4bb8d66709e6ad/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:18.446411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8fdd5-c4a1-44e1-845f-8586be356471 None admin] RBAC: Authorizing `identity:create_grant(project_id=4debe830e03247c79d3889e7479b9dfa, user_id=72cd343fe8fe49438f4bb8d66709e6ad, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:18.448123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8fdd5-c4a1-44e1-845f-8586be356471 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:18.480394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 66/134] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:39:18 2022] PUT /identity/v3/projects/4debe830e03247c79d3889e7479b9dfa/users/72cd343fe8fe49438f4bb8d66709e6ad/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:18.481657 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8fdd5-c4a1-44e1-845f-8586be356471 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:18.491616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521a40b-34b7-47ba-930b-bd46200db1ac None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:18.517175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:18.527254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e4d5f-a43a-416f-8d36-17b315fea2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:18.528727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e4d5f-a43a-416f-8d36-17b315fea2a2 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:18.529341 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e4d5f-a43a-416f-8d36-17b315fea2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:18.529996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e4d5f-a43a-416f-8d36-17b315fea2a2 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:18.537443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e4d5f-a43a-416f-8d36-17b315fea2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:18.539246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e4d5f-a43a-416f-8d36-17b315fea2a2 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:18.541405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 69/135] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:18 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:18.553612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3560b-6911-449f-ba7e-d4672c49fc14 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:18.553612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3560b-6911-449f-ba7e-d4672c49fc14 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:18.553612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3560b-6911-449f-ba7e-d4672c49fc14 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:18.553612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 70/136] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:18.553612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3560b-6911-449f-ba7e-d4672c49fc14 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:18.660351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:18.673162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8fdd5-c4a1-44e1-845f-8586be356471 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:18.733950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343667e-be31-4675-afd1-73d2483a9463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:18.735078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343667e-be31-4675-afd1-73d2483a9463 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:18.735663 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343667e-be31-4675-afd1-73d2483a9463 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:18.736088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343667e-be31-4675-afd1-73d2483a9463 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460/users/9d09bdd9927e44ffbfd9b2ec447a5887/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:18.739848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343667e-be31-4675-afd1-73d2483a9463 None admin] RBAC: Authorizing `identity:create_grant(project_id=574afa1590854c65a807df4589e2c460, user_id=9d09bdd9927e44ffbfd9b2ec447a5887, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:18.742020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343667e-be31-4675-afd1-73d2483a9463 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:18.749639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3560b-6911-449f-ba7e-d4672c49fc14 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:18.760668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 67/137] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:39:18 2022] PUT /identity/v3/projects/574afa1590854c65a807df4589e2c460/users/9d09bdd9927e44ffbfd9b2ec447a5887/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:18.762115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b1a5-d506-474e-b9f4-5281c041c81d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:18.762405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b1a5-d506-474e-b9f4-5281c041c81d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:18.762731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b1a5-d506-474e-b9f4-5281c041c81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:18.785317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9912b1a5-d506-474e-b9f4-5281c041c81d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:18.786597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9912b1a5-d506-474e-b9f4-5281c041c81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:18.824624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f9742-3327-41aa-8f72-4aa785457cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:18.825572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f9742-3327-41aa-8f72-4aa785457cb3 None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:18.825847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f9742-3327-41aa-8f72-4aa785457cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:18.826122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f9742-3327-41aa-8f72-4aa785457cb3 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460/users/72cd343fe8fe49438f4bb8d66709e6ad/roles/b127031d393048d7ae9a35015113e82a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:18.836890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f9742-3327-41aa-8f72-4aa785457cb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=574afa1590854c65a807df4589e2c460, user_id=72cd343fe8fe49438f4bb8d66709e6ad, role_id=b127031d393048d7ae9a35015113e82a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:18.838560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f9742-3327-41aa-8f72-4aa785457cb3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:18.868319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 68/138] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:18.883269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 71/139] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:39:18 2022] PUT /identity/v3/projects/574afa1590854c65a807df4589e2c460/users/72cd343fe8fe49438f4bb8d66709e6ad/roles/b127031d393048d7ae9a35015113e82a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:18.896142 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912b1a5-d506-474e-b9f4-5281c041c81d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:19.000554 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:19.014797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912b1a5-d506-474e-b9f4-5281c041c81d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:19.054840 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08e0b6-3c91-4568-8482-fa30e10aff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:19.055569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08e0b6-3c91-4568-8482-fa30e10aff4d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:19.055851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08e0b6-3c91-4568-8482-fa30e10aff4d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:19.056123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08e0b6-3c91-4568-8482-fa30e10aff4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:19.057380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08e0b6-3c91-4568-8482-fa30e10aff4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:19.059090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08e0b6-3c91-4568-8482-fa30e10aff4d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:19.060905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 69/140] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:19.066796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f9742-3327-41aa-8f72-4aa785457cb3 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:19.129265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:19.181389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f9742-3327-41aa-8f72-4aa785457cb3 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:19.219338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017693c-70b3-4518-9cff-14735246b4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:19.220040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017693c-70b3-4518-9cff-14735246b4bd None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:19.220320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017693c-70b3-4518-9cff-14735246b4bd None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:19.220591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017693c-70b3-4518-9cff-14735246b4bd None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:19.221968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017693c-70b3-4518-9cff-14735246b4bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:19.223515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017693c-70b3-4518-9cff-14735246b4bd None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:19.251457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 72/141] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:39:19 2022] POST /identity/v3/users => generated 283 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:20.253277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ccb00-42ff-4a83-becd-b0ce785b2f9c None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:20.253834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ccb00-42ff-4a83-becd-b0ce785b2f9c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:20.253917 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ccb00-42ff-4a83-becd-b0ce785b2f9c None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:20.255267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 70/142] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:20.260806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de34e3-7be3-431c-84ed-dc98bba645e8 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:20.261237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de34e3-7be3-431c-84ed-dc98bba645e8 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:20.261420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de34e3-7be3-431c-84ed-dc98bba645e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:20.280169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-b9de34e3-7be3-431c-84ed-dc98bba645e8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:20.281383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b9de34e3-7be3-431c-84ed-dc98bba645e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:20.366204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 73/143] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:20.388981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ccb00-42ff-4a83-becd-b0ce785b2f9c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:20.477561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc5c87-ebb3-413a-bab0-6ae9e77a7d00 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:20.477916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc5c87-ebb3-413a-bab0-6ae9e77a7d00 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:20.478130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc5c87-ebb3-413a-bab0-6ae9e77a7d00 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:20.479508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 74/144] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:20.495331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d57124-79a1-47d9-8702-07eebcca6650 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:20.495613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d57124-79a1-47d9-8702-07eebcca6650 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:20.495884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d57124-79a1-47d9-8702-07eebcca6650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:20.519591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-e5d57124-79a1-47d9-8702-07eebcca6650 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:20.520682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e5d57124-79a1-47d9-8702-07eebcca6650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:20.523284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ccb00-42ff-4a83-becd-b0ce785b2f9c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:20.572975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49792729-4188-4633-b079-54b6be208c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:20.573806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49792729-4188-4633-b079-54b6be208c16 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:20.574085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49792729-4188-4633-b079-54b6be208c16 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:20.574357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49792729-4188-4633-b079-54b6be208c16 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:20.585865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49792729-4188-4633-b079-54b6be208c16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-49792729-4188-4633-b079-54b6be208c16 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:20.595923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:20.597665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:20.598509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:20.599184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 71/145] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:20.604640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49792729-4188-4633-b079-54b6be208c16 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:20.607485 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:20.626965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:20.703884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 75/146] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:20.725797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49792729-4188-4633-b079-54b6be208c16 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:20.746677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d57124-79a1-47d9-8702-07eebcca6650 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:20.762611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c68804-716c-463c-993e-6f0a857fe11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:20.763428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c68804-716c-463c-993e-6f0a857fe11d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:20.763710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c68804-716c-463c-993e-6f0a857fe11d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:20.764107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c68804-716c-463c-993e-6f0a857fe11d None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:20.764979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c68804-716c-463c-993e-6f0a857fe11d None admin] RBAC: Adding query filter params (name=reader) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:20.765289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c68804-716c-463c-993e-6f0a857fe11d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:20.767288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c68804-716c-463c-993e-6f0a857fe11d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:20.777110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 72/147] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:20.782880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c68804-716c-463c-993e-6f0a857fe11d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:20.923261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:20.985990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c68804-716c-463c-993e-6f0a857fe11d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:21.035867 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d57124-79a1-47d9-8702-07eebcca6650 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:21.078979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3087e8-7865-49f7-8577-63179815296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:21.080286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3087e8-7865-49f7-8577-63179815296a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:21.080800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3087e8-7865-49f7-8577-63179815296a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:21.081299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3087e8-7865-49f7-8577-63179815296a None admin] PATH_INFO: `/v3/users/c26dec6c07a148479302cc9f2e861a24` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:21.082836 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3087e8-7865-49f7-8577-63179815296a None admin] RBAC: Authorizing `identity:get_user(user_id=c26dec6c07a148479302cc9f2e861a24)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:21.084554 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3087e8-7865-49f7-8577-63179815296a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:21.086479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 73/148] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:20 2022] GET /identity/v3/users/c26dec6c07a148479302cc9f2e861a24 => generated 289 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:21.092917 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3087e8-7865-49f7-8577-63179815296a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:21.131407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcd196-1fa6-4a55-b6b1-59e72727e584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:21.132233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcd196-1fa6-4a55-b6b1-59e72727e584 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:21.132510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcd196-1fa6-4a55-b6b1-59e72727e584 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:21.132787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcd196-1fa6-4a55-b6b1-59e72727e584 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:21.166535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcd196-1fa6-4a55-b6b1-59e72727e584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6ddcd196-1fa6-4a55-b6b1-59e72727e584 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:21.175263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:21.176176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:21.176926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 76/149] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:21.192360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcd196-1fa6-4a55-b6b1-59e72727e584 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:21.294995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:21.303537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3087e8-7865-49f7-8577-63179815296a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:21.398874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193547a-2435-44ea-95b0-5390a762943b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:21.399777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193547a-2435-44ea-95b0-5390a762943b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:21.400051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193547a-2435-44ea-95b0-5390a762943b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:21.400321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193547a-2435-44ea-95b0-5390a762943b None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:21.401528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193547a-2435-44ea-95b0-5390a762943b None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:21.403017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193547a-2435-44ea-95b0-5390a762943b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:21.404916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 74/150] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:21 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:21.412710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193547a-2435-44ea-95b0-5390a762943b None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:21.461033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:21.499553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcd196-1fa6-4a55-b6b1-59e72727e584 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:21.562822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f804e7-6b0e-42f9-a425-cb181fb5c7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:21.563787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f804e7-6b0e-42f9-a425-cb181fb5c7d6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:21.564084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f804e7-6b0e-42f9-a425-cb181fb5c7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:21.564360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f804e7-6b0e-42f9-a425-cb181fb5c7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:21.565326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f804e7-6b0e-42f9-a425-cb181fb5c7d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:21.565662 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f804e7-6b0e-42f9-a425-cb181fb5c7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:21.567409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f804e7-6b0e-42f9-a425-cb181fb5c7d6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:21.580300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 77/151] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:21.588837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f804e7-6b0e-42f9-a425-cb181fb5c7d6 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:21.664473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193547a-2435-44ea-95b0-5390a762943b None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:21.703801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7abb1-7b43-4da1-ac14-b8b33693e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:21.704693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7abb1-7b43-4da1-ac14-b8b33693e905 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:21.704971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7abb1-7b43-4da1-ac14-b8b33693e905 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:21.705243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7abb1-7b43-4da1-ac14-b8b33693e905 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460/users/c26dec6c07a148479302cc9f2e861a24/roles/0c5069e08baa4820b7154564a801ba7e` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:21.708537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7abb1-7b43-4da1-ac14-b8b33693e905 None admin] RBAC: Authorizing `identity:create_grant(project_id=574afa1590854c65a807df4589e2c460, user_id=c26dec6c07a148479302cc9f2e861a24, role_id=0c5069e08baa4820b7154564a801ba7e)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:21.710034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7abb1-7b43-4da1-ac14-b8b33693e905 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:21.727597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 75/152] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:39:21 2022] PUT /identity/v3/projects/574afa1590854c65a807df4589e2c460/users/c26dec6c07a148479302cc9f2e861a24/roles/0c5069e08baa4820b7154564a801ba7e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:39:21.757915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:21.768046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f804e7-6b0e-42f9-a425-cb181fb5c7d6 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:21.823196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9984c-7ceb-4f81-93c7-5aac788e94a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:21.824095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9984c-7ceb-4f81-93c7-5aac788e94a5 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:21.824378 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9984c-7ceb-4f81-93c7-5aac788e94a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:21.824655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9984c-7ceb-4f81-93c7-5aac788e94a5 None admin] PATH_INFO: `/v3/users/72cd343fe8fe49438f4bb8d66709e6ad` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:21.825935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9984c-7ceb-4f81-93c7-5aac788e94a5 None admin] RBAC: Authorizing `identity:get_user(user_id=72cd343fe8fe49438f4bb8d66709e6ad)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:21.827411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9984c-7ceb-4f81-93c7-5aac788e94a5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:21.829221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 78/153] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:21 2022] GET /identity/v3/users/72cd343fe8fe49438f4bb8d66709e6ad => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:21.835518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7abb1-7b43-4da1-ac14-b8b33693e905 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:21.970036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:21.981937 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7abb1-7b43-4da1-ac14-b8b33693e905 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:22.036130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc33c1-17fc-4ee3-b94a-e91a9ca5d2e4 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:22.036688 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc33c1-17fc-4ee3-b94a-e91a9ca5d2e4 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:22.036947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc33c1-17fc-4ee3-b94a-e91a9ca5d2e4 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:22.038365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 79/154] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:22.043155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de86955-265b-436b-bda1-9e54aab06cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:22.043813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de86955-265b-436b-bda1-9e54aab06cce None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:22.044095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de86955-265b-436b-bda1-9e54aab06cce None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:22.044369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de86955-265b-436b-bda1-9e54aab06cce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:22.059117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8db9-6a3d-4405-b9bd-4df640d091e8 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:22.059400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8db9-6a3d-4405-b9bd-4df640d091e8 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:22.059678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8db9-6a3d-4405-b9bd-4df640d091e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:22.090984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-3a4f8db9-6a3d-4405-b9bd-4df640d091e8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:22.092135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3a4f8db9-6a3d-4405-b9bd-4df640d091e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:22.149407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de86955-265b-436b-bda1-9e54aab06cce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:22.151051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de86955-265b-436b-bda1-9e54aab06cce None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:22.229478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 80/155] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:22.234736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:22.264654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 76/156] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:22.270325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8db9-6a3d-4405-b9bd-4df640d091e8 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:22.278965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de86955-265b-436b-bda1-9e54aab06cce None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:22.397839 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de86955-265b-436b-bda1-9e54aab06cce None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:22.418671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:22.430688 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f8db9-6a3d-4405-b9bd-4df640d091e8 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:22.446807 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9abb82-d4e6-44c3-84e3-d6142ed0eb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:22.447790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9abb82-d4e6-44c3-84e3-d6142ed0eb39 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:22.448075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9abb82-d4e6-44c3-84e3-d6142ed0eb39 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:22.448351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9abb82-d4e6-44c3-84e3-d6142ed0eb39 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:22.450367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9abb82-d4e6-44c3-84e3-d6142ed0eb39 None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:22.452054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9abb82-d4e6-44c3-84e3-d6142ed0eb39 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:22.454032 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 77/157] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:22 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:22.461181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9abb82-d4e6-44c3-84e3-d6142ed0eb39 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:22.490420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b653d-2e94-47aa-803f-57401bdf00fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:22.491259 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b653d-2e94-47aa-803f-57401bdf00fb None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:22.491543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b653d-2e94-47aa-803f-57401bdf00fb None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:22.491825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b653d-2e94-47aa-803f-57401bdf00fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:22.516548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b653d-2e94-47aa-803f-57401bdf00fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-520b653d-2e94-47aa-803f-57401bdf00fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:22.525334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:22.526321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:22.527071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 81/158] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:22.532348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b653d-2e94-47aa-803f-57401bdf00fb None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:22.546241 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:22.655821 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9abb82-d4e6-44c3-84e3-d6142ed0eb39 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:22.679441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b653d-2e94-47aa-803f-57401bdf00fb None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:22.734944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fa376-6ba7-4d70-8c4d-302f63ff2f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:22.735630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fa376-6ba7-4d70-8c4d-302f63ff2f5d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:22.735914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fa376-6ba7-4d70-8c4d-302f63ff2f5d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:22.736191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fa376-6ba7-4d70-8c4d-302f63ff2f5d None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:22.737014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fa376-6ba7-4d70-8c4d-302f63ff2f5d None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:22.737319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fa376-6ba7-4d70-8c4d-302f63ff2f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:22.738878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fa376-6ba7-4d70-8c4d-302f63ff2f5d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:22.739155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dcca1-f67c-45de-9347-e02fb8568894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:22.739805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dcca1-f67c-45de-9347-e02fb8568894 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:22.740083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dcca1-f67c-45de-9347-e02fb8568894 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:22.740356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dcca1-f67c-45de-9347-e02fb8568894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:22.741192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547dcca1-f67c-45de-9347-e02fb8568894 None admin] RBAC: Adding query filter params (role.id=2e1db0cf27684f389b9320673721684d, scope.project.id=574afa1590854c65a807df4589e2c460, user.id=72cd343fe8fe49438f4bb8d66709e6ad) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:22.741496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547dcca1-f67c-45de-9347-e02fb8568894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:22.742999 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547dcca1-f67c-45de-9347-e02fb8568894 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:22.754069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 82/159] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:22.755162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d97f4-e205-4e68-891f-7baa6d54b37e None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:22.755452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d97f4-e205-4e68-891f-7baa6d54b37e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:22.755732 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d97f4-e205-4e68-891f-7baa6d54b37e None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:22.757242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 83/160] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:22.758990 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d97f4-e205-4e68-891f-7baa6d54b37e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:22.760019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 78/161] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:22 2022] GET /identity/v3/role_assignments?user.id=72cd343fe8fe49438f4bb8d66709e6ad&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=2e1db0cf27684f389b9320673721684d => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:22.771761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0342b-1eb9-4919-b6bd-c0ada1205338 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:22.772082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0342b-1eb9-4919-b6bd-c0ada1205338 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:22.772311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0342b-1eb9-4919-b6bd-c0ada1205338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:22.784994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:22.806623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-97d0342b-1eb9-4919-b6bd-c0ada1205338 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:22.807820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-97d0342b-1eb9-4919-b6bd-c0ada1205338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:22.926455 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d97f4-e205-4e68-891f-7baa6d54b37e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:22.967222 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 79/162] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:22.979439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e16ed-b558-4d6a-be67-66073db44fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:22.980367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e16ed-b558-4d6a-be67-66073db44fdc None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:22.980645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e16ed-b558-4d6a-be67-66073db44fdc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:22.980913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e16ed-b558-4d6a-be67-66073db44fdc None admin] PATH_INFO: `/v3/users/72cd343fe8fe49438f4bb8d66709e6ad` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:22.982218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e16ed-b558-4d6a-be67-66073db44fdc None admin] RBAC: Authorizing `identity:get_user(user_id=72cd343fe8fe49438f4bb8d66709e6ad)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:22.983706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e16ed-b558-4d6a-be67-66073db44fdc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:22.985692 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 84/163] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:22 2022] GET /identity/v3/users/72cd343fe8fe49438f4bb8d66709e6ad => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:22.992376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0342b-1eb9-4919-b6bd-c0ada1205338 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:23.032174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e16ed-b558-4d6a-be67-66073db44fdc None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:23.051337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:23.168754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0342b-1eb9-4919-b6bd-c0ada1205338 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:23.221797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46e025-5b42-46d0-b041-32cfef9c48c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:23.222472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46e025-5b42-46d0-b041-32cfef9c48c8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:23.222750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46e025-5b42-46d0-b041-32cfef9c48c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:23.223025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46e025-5b42-46d0-b041-32cfef9c48c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:23.223543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:23.265205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e16ed-b558-4d6a-be67-66073db44fdc None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:23.320578 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46e025-5b42-46d0-b041-32cfef9c48c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:23.326164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46e025-5b42-46d0-b041-32cfef9c48c8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:23.351222 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307ddc9-670b-4f31-a58e-4ca3f31a657b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:23.352042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307ddc9-670b-4f31-a58e-4ca3f31a657b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:23.352316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307ddc9-670b-4f31-a58e-4ca3f31a657b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:23.352584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307ddc9-670b-4f31-a58e-4ca3f31a657b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:23.371809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:23.382579 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307ddc9-670b-4f31-a58e-4ca3f31a657b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-d307ddc9-670b-4f31-a58e-4ca3f31a657b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:23.390435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:23.391284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:23.392045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 85/164] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:39:23.415022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307ddc9-670b-4f31-a58e-4ca3f31a657b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:23.471924 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 80/165] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:23.478368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46e025-5b42-46d0-b041-32cfef9c48c8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:23.625842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46e025-5b42-46d0-b041-32cfef9c48c8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:23.677604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce56cc-fb89-45fc-b896-f7479a8d16e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:23.678622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce56cc-fb89-45fc-b896-f7479a8d16e5 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:23.678909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce56cc-fb89-45fc-b896-f7479a8d16e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:23.679181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce56cc-fb89-45fc-b896-f7479a8d16e5 None admin] PATH_INFO: `/v3/projects/4debe830e03247c79d3889e7479b9dfa` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:23.683107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce56cc-fb89-45fc-b896-f7479a8d16e5 None admin] RBAC: Authorizing `identity:get_project(project_id=4debe830e03247c79d3889e7479b9dfa)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:23.684591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce56cc-fb89-45fc-b896-f7479a8d16e5 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:23.686823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 81/166] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:23 2022] GET /identity/v3/projects/4debe830e03247c79d3889e7479b9dfa => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:23.688106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937d552-1666-4ccb-afe6-665038b96e7c None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:23.688393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937d552-1666-4ccb-afe6-665038b96e7c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:23.688667 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937d552-1666-4ccb-afe6-665038b96e7c None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:23.690466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 82/167] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:23.693304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937d552-1666-4ccb-afe6-665038b96e7c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:23.697885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:23.723954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307ddc9-670b-4f31-a58e-4ca3f31a657b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:23.809805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937d552-1666-4ccb-afe6-665038b96e7c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:23.810029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba79858-a614-468f-9999-8fd31092acd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:23.810531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba79858-a614-468f-9999-8fd31092acd6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:23.810816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba79858-a614-468f-9999-8fd31092acd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:23.811093 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba79858-a614-468f-9999-8fd31092acd6 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:23.811859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba79858-a614-468f-9999-8fd31092acd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:23.812164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba79858-a614-468f-9999-8fd31092acd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:23.814040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba79858-a614-468f-9999-8fd31092acd6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:23.837280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 86/168] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:23.837280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8e50d-0c59-46d2-b4bf-38b799e7c730 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:23.837280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8e50d-0c59-46d2-b4bf-38b799e7c730 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:23.837280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8e50d-0c59-46d2-b4bf-38b799e7c730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:23.846825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:23.859921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c23f44-ee21-481d-a9bb-a011abc09fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:23.860800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c23f44-ee21-481d-a9bb-a011abc09fe9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:23.861330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c23f44-ee21-481d-a9bb-a011abc09fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:23.862290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c23f44-ee21-481d-a9bb-a011abc09fe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:23.863104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-0ce8e50d-0c59-46d2-b4bf-38b799e7c730 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:23.863445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c23f44-ee21-481d-a9bb-a011abc09fe9 None admin] RBAC: Adding query filter params (role.id=2e1db0cf27684f389b9320673721684d, scope.project.id=4debe830e03247c79d3889e7479b9dfa, user.id=72cd343fe8fe49438f4bb8d66709e6ad) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:23.863924 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c23f44-ee21-481d-a9bb-a011abc09fe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:23.864316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0ce8e50d-0c59-46d2-b4bf-38b799e7c730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:23.865714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c23f44-ee21-481d-a9bb-a011abc09fe9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:23.872873 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 83/169] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:23 2022] GET /identity/v3/role_assignments?user.id=72cd343fe8fe49438f4bb8d66709e6ad&scope.project.id=4debe830e03247c79d3889e7479b9dfa&role.id=2e1db0cf27684f389b9320673721684d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:23.873948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c23f44-ee21-481d-a9bb-a011abc09fe9 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:23.949014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:23.990985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c23f44-ee21-481d-a9bb-a011abc09fe9 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:24.029846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb214e-80b6-43f7-9023-b67514e3f595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:24.030968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb214e-80b6-43f7-9023-b67514e3f595 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:24.031417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb214e-80b6-43f7-9023-b67514e3f595 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:24.031833 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb214e-80b6-43f7-9023-b67514e3f595 None admin] PATH_INFO: `/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:24.033200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb214e-80b6-43f7-9023-b67514e3f595 None admin] RBAC: Authorizing `identity:get_user(user_id=9d09bdd9927e44ffbfd9b2ec447a5887)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:24.035434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb214e-80b6-43f7-9023-b67514e3f595 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:24.036622 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 87/170] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:24.037811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 84/171] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:23 2022] GET /identity/v3/users/9d09bdd9927e44ffbfd9b2ec447a5887 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:24.043493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8e50d-0c59-46d2-b4bf-38b799e7c730 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:24.075719 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb214e-80b6-43f7-9023-b67514e3f595 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:24.209605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb214e-80b6-43f7-9023-b67514e3f595 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:24.248558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec4a6d-bbda-4f46-9ad6-1f68c5db13e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:24.249692 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4a6d-bbda-4f46-9ad6-1f68c5db13e4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:24.250150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4a6d-bbda-4f46-9ad6-1f68c5db13e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:24.250620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4a6d-bbda-4f46-9ad6-1f68c5db13e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:24.253805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:24.263823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec4a6d-bbda-4f46-9ad6-1f68c5db13e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:24.267705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8e50d-0c59-46d2-b4bf-38b799e7c730 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-feec4a6d-bbda-4f46-9ad6-1f68c5db13e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:24.273600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:24.274467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:24.275310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:24.277860 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 85/172] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:24.279465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3ef1e-c7b3-41c9-b157-e1f7ff805304 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:24.279893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3ef1e-c7b3-41c9-b157-e1f7ff805304 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:24.280309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3ef1e-c7b3-41c9-b157-e1f7ff805304 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:24.281773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 86/173] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:24.294597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3ef1e-c7b3-41c9-b157-e1f7ff805304 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:24.326569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf86dd2-5a9b-4f9c-a4e3-2b8e009c8a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:24.328536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf86dd2-5a9b-4f9c-a4e3-2b8e009c8a2d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:24.328825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf86dd2-5a9b-4f9c-a4e3-2b8e009c8a2d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:24.329108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf86dd2-5a9b-4f9c-a4e3-2b8e009c8a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:24.341930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:24.437717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf86dd2-5a9b-4f9c-a4e3-2b8e009c8a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:24.439244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf86dd2-5a9b-4f9c-a4e3-2b8e009c8a2d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:24.466279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3ef1e-c7b3-41c9-b157-e1f7ff805304 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:24.520159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048ddab-f9e8-4fda-92f1-131bb69a2d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:24.521518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048ddab-f9e8-4fda-92f1-131bb69a2d89 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:24.522645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048ddab-f9e8-4fda-92f1-131bb69a2d89 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:24.522925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048ddab-f9e8-4fda-92f1-131bb69a2d89 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:24.523684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048ddab-f9e8-4fda-92f1-131bb69a2d89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:24.523990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048ddab-f9e8-4fda-92f1-131bb69a2d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:24.525402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048ddab-f9e8-4fda-92f1-131bb69a2d89 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:24.536135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 87/174] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:24.542098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f3d59-9275-49df-b911-61b063e53bc3 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:24.542419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f3d59-9275-49df-b911-61b063e53bc3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:24.542707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f3d59-9275-49df-b911-61b063e53bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:24.568797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7c0f3d59-9275-49df-b911-61b063e53bc3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:24.570134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7c0f3d59-9275-49df-b911-61b063e53bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:24.574819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 88/175] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:24.576365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afad08-e629-467e-9b1a-cf4a80c8ea5d None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:24.576868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afad08-e629-467e-9b1a-cf4a80c8ea5d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:24.577355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afad08-e629-467e-9b1a-cf4a80c8ea5d None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:24.578860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 89/176] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:24.579749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afad08-e629-467e-9b1a-cf4a80c8ea5d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:24.625669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:24.651992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:24.675610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 88/177] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:24.676766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f3d59-9275-49df-b911-61b063e53bc3 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:24.708733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afad08-e629-467e-9b1a-cf4a80c8ea5d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:24.751545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33051bcf-aa03-4464-9190-ec38a79f313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:24.754722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33051bcf-aa03-4464-9190-ec38a79f313e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:24.755234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33051bcf-aa03-4464-9190-ec38a79f313e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:24.755699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33051bcf-aa03-4464-9190-ec38a79f313e None admin] PATH_INFO: `/v3/users/72cd343fe8fe49438f4bb8d66709e6ad` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:24.757134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33051bcf-aa03-4464-9190-ec38a79f313e None admin] RBAC: Authorizing `identity:get_user(user_id=72cd343fe8fe49438f4bb8d66709e6ad)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:24.759279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33051bcf-aa03-4464-9190-ec38a79f313e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:24.761541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 90/178] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:24 2022] GET /identity/v3/users/72cd343fe8fe49438f4bb8d66709e6ad => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:24.763302 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c8668-0d26-4a29-aa63-1ec0cfc4d279 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:24.764232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c8668-0d26-4a29-aa63-1ec0cfc4d279 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:24.764715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c8668-0d26-4a29-aa63-1ec0cfc4d279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:24.785349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-aa4c8668-0d26-4a29-aa63-1ec0cfc4d279 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:24.786871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-aa4c8668-0d26-4a29-aa63-1ec0cfc4d279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:24.788884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f3d59-9275-49df-b911-61b063e53bc3 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:24.832346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19030b-6294-418b-8920-3d3f0f2bbd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:24.833956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19030b-6294-418b-8920-3d3f0f2bbd2e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:24.834442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19030b-6294-418b-8920-3d3f0f2bbd2e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:24.835192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19030b-6294-418b-8920-3d3f0f2bbd2e None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:24.836883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc19030b-6294-418b-8920-3d3f0f2bbd2e None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:24.839328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc19030b-6294-418b-8920-3d3f0f2bbd2e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:24.841755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 89/179] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:24 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:24.842720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19030b-6294-418b-8920-3d3f0f2bbd2e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:24.865470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 91/180] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:24.871072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c8668-0d26-4a29-aa63-1ec0cfc4d279 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:24.911753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:24.932135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:24.988295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19030b-6294-418b-8920-3d3f0f2bbd2e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:24.990852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c8668-0d26-4a29-aa63-1ec0cfc4d279 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:25.027649 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b1d0d-c3dc-4068-b004-af46b249f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:25.028300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b1d0d-c3dc-4068-b004-af46b249f40f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.028575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b1d0d-c3dc-4068-b004-af46b249f40f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.028848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b1d0d-c3dc-4068-b004-af46b249f40f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.031992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875972a0-fb3e-401e-8953-f26b66b3cbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:25.034348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875972a0-fb3e-401e-8953-f26b66b3cbae None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.035093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875972a0-fb3e-401e-8953-f26b66b3cbae None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.035992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875972a0-fb3e-401e-8953-f26b66b3cbae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.052414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875972a0-fb3e-401e-8953-f26b66b3cbae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-875972a0-fb3e-401e-8953-f26b66b3cbae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:25.063551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:25.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:25.065272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:25.066997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 90/181] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:25.068550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875972a0-fb3e-401e-8953-f26b66b3cbae None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:25.106943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b1d0d-c3dc-4068-b004-af46b249f40f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:25.108469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b1d0d-c3dc-4068-b004-af46b249f40f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:25.140017 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:25.176344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875972a0-fb3e-401e-8953-f26b66b3cbae None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:25.192342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:25.211114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0d2d2-7836-4c84-bf9d-5e04fb4c2baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:25.211774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0d2d2-7836-4c84-bf9d-5e04fb4c2baf None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.212057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0d2d2-7836-4c84-bf9d-5e04fb4c2baf None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.212336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0d2d2-7836-4c84-bf9d-5e04fb4c2baf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.213165 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0d2d2-7836-4c84-bf9d-5e04fb4c2baf None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=574afa1590854c65a807df4589e2c460, user.id=9d09bdd9927e44ffbfd9b2ec447a5887) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:25.213476 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0d2d2-7836-4c84-bf9d-5e04fb4c2baf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:25.214947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0d2d2-7836-4c84-bf9d-5e04fb4c2baf None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:25.215791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 92/182] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:25.216627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b1d0d-c3dc-4068-b004-af46b249f40f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:25.221119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 91/183] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:25 2022] GET /identity/v3/role_assignments?user.id=9d09bdd9927e44ffbfd9b2ec447a5887&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:25.222711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0d2d2-7836-4c84-bf9d-5e04fb4c2baf None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:25.328952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b1d0d-c3dc-4068-b004-af46b249f40f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:25.340750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0d2d2-7836-4c84-bf9d-5e04fb4c2baf None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:25.366583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:25.373298 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1dd184-33d3-4006-bdb9-c1d94f46122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:25.374397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1dd184-33d3-4006-bdb9-c1d94f46122c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.374675 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1dd184-33d3-4006-bdb9-c1d94f46122c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.374945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1dd184-33d3-4006-bdb9-c1d94f46122c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.385067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af29d0-069b-47b1-b88b-f71ee66f9423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:25.385977 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af29d0-069b-47b1-b88b-f71ee66f9423 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.386175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af29d0-069b-47b1-b88b-f71ee66f9423 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.386430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af29d0-069b-47b1-b88b-f71ee66f9423 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.387194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af29d0-069b-47b1-b88b-f71ee66f9423 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:25.388146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af29d0-069b-47b1-b88b-f71ee66f9423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:25.388258 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1dd184-33d3-4006-bdb9-c1d94f46122c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:25.389246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af29d0-069b-47b1-b88b-f71ee66f9423 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-1f1dd184-33d3-4006-bdb9-c1d94f46122c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:25.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:25.397686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:25.398483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:25.401738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 93/184] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:25.403043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 92/185] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:25.403161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8abf18-a47d-4d7c-a8c6-b109e8a9cb2a None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.403398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8abf18-a47d-4d7c-a8c6-b109e8a9cb2a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.403681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8abf18-a47d-4d7c-a8c6-b109e8a9cb2a None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.404973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2eff5b-1549-4b96-a258-ae04d7252cc0 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.404973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2eff5b-1549-4b96-a258-ae04d7252cc0 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.404973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2eff5b-1549-4b96-a258-ae04d7252cc0 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.405195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 94/186] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:25.409639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8abf18-a47d-4d7c-a8c6-b109e8a9cb2a None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:25.409773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 93/187] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:25.411959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2eff5b-1549-4b96-a258-ae04d7252cc0 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:25.462485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:25.522714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2eff5b-1549-4b96-a258-ae04d7252cc0 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:25.527253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8abf18-a47d-4d7c-a8c6-b109e8a9cb2a None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:25.556917 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209ffa6-624e-48de-b1b3-d8692e176240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:25.557591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209ffa6-624e-48de-b1b3-d8692e176240 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.557881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209ffa6-624e-48de-b1b3-d8692e176240 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.558157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209ffa6-624e-48de-b1b3-d8692e176240 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.558904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209ffa6-624e-48de-b1b3-d8692e176240 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:25.559210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209ffa6-624e-48de-b1b3-d8692e176240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:25.560609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209ffa6-624e-48de-b1b3-d8692e176240 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:25.562604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513d63f-df64-40a8-be68-d1ab7ad9efce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:25.563467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513d63f-df64-40a8-be68-d1ab7ad9efce None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.563745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513d63f-df64-40a8-be68-d1ab7ad9efce None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.564021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513d63f-df64-40a8-be68-d1ab7ad9efce None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.565774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513d63f-df64-40a8-be68-d1ab7ad9efce None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:25.567210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513d63f-df64-40a8-be68-d1ab7ad9efce None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:25.568929 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 95/188] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:25 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:25.569724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513d63f-df64-40a8-be68-d1ab7ad9efce None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:25.574307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 94/189] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:25.574307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54128a4e-f8f5-4c5f-bc3c-6bac13462cb6 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.574307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54128a4e-f8f5-4c5f-bc3c-6bac13462cb6 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.574307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54128a4e-f8f5-4c5f-bc3c-6bac13462cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.590039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-54128a4e-f8f5-4c5f-bc3c-6bac13462cb6 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:25.591158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-54128a4e-f8f5-4c5f-bc3c-6bac13462cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:25.634014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:25.660584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 95/190] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:25.663021 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e0b40-466d-4b36-ba34-4f58e08b8ddc None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.663312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e0b40-466d-4b36-ba34-4f58e08b8ddc None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.663586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e0b40-466d-4b36-ba34-4f58e08b8ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.682585 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-b64e0b40-466d-4b36-ba34-4f58e08b8ddc None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:25.683135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513d63f-df64-40a8-be68-d1ab7ad9efce None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:25.684331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b64e0b40-466d-4b36-ba34-4f58e08b8ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:25.717361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:25.717528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bf350-21ad-469b-836a-a1c8ae11a071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:25.718363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bf350-21ad-469b-836a-a1c8ae11a071 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.718644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bf350-21ad-469b-836a-a1c8ae11a071 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.718914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bf350-21ad-469b-836a-a1c8ae11a071 None admin] PATH_INFO: `/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.733129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bf350-21ad-469b-836a-a1c8ae11a071 None admin] RBAC: Authorizing `identity:get_user(user_id=0d46c4c47bfb4ad0b2c58eca1669dcad)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:25.734640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bf350-21ad-469b-836a-a1c8ae11a071 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:25.736337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 96/191] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:25 2022] GET /identity/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:25.737768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bf350-21ad-469b-836a-a1c8ae11a071 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:25.762428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 96/192] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:25.763267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e0b40-466d-4b36-ba34-4f58e08b8ddc None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:25.836928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bf350-21ad-469b-836a-a1c8ae11a071 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:25.854471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:25.863035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e0b40-466d-4b36-ba34-4f58e08b8ddc None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:25.871600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe09dc4-a19f-4aee-9ab1-177c4e16079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:25.872256 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe09dc4-a19f-4aee-9ab1-177c4e16079e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.872538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe09dc4-a19f-4aee-9ab1-177c4e16079e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.872816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe09dc4-a19f-4aee-9ab1-177c4e16079e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.873683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe09dc4-a19f-4aee-9ab1-177c4e16079e None admin] RBAC: Adding query filter params (role.id=b127031d393048d7ae9a35015113e82a, scope.project.id=574afa1590854c65a807df4589e2c460, user.id=72cd343fe8fe49438f4bb8d66709e6ad) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:25.873997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe09dc4-a19f-4aee-9ab1-177c4e16079e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:25.875490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe09dc4-a19f-4aee-9ab1-177c4e16079e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:25.881226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 97/193] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:25 2022] GET /identity/v3/role_assignments?user.id=72cd343fe8fe49438f4bb8d66709e6ad&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=b127031d393048d7ae9a35015113e82a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:25.882036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe09dc4-a19f-4aee-9ab1-177c4e16079e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:25.898130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b723cf1-24d4-47a3-ae14-0c2195c4843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:25.899033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b723cf1-24d4-47a3-ae14-0c2195c4843a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:25.899315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b723cf1-24d4-47a3-ae14-0c2195c4843a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:25.899590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b723cf1-24d4-47a3-ae14-0c2195c4843a None admin] PATH_INFO: `/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:25.900892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b723cf1-24d4-47a3-ae14-0c2195c4843a None admin] RBAC: Authorizing `identity:get_user(user_id=0d46c4c47bfb4ad0b2c58eca1669dcad)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:25.902428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b723cf1-24d4-47a3-ae14-0c2195c4843a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:25.904190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 97/194] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:25 2022] GET /identity/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:25.905005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b723cf1-24d4-47a3-ae14-0c2195c4843a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:25.942249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:26.014990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b723cf1-24d4-47a3-ae14-0c2195c4843a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:26.024635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe09dc4-a19f-4aee-9ab1-177c4e16079e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:26.057741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4503e8d-8df7-48d4-bfa2-0d227dd3c305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:26.057741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4503e8d-8df7-48d4-bfa2-0d227dd3c305 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:26.057741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4503e8d-8df7-48d4-bfa2-0d227dd3c305 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:26.058082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4503e8d-8df7-48d4-bfa2-0d227dd3c305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:26.069995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97447ef2-9dc8-4379-8a60-0003ae931912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:26.071139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97447ef2-9dc8-4379-8a60-0003ae931912 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:26.071139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97447ef2-9dc8-4379-8a60-0003ae931912 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:26.071315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97447ef2-9dc8-4379-8a60-0003ae931912 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:26.072904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97447ef2-9dc8-4379-8a60-0003ae931912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:26.074742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97447ef2-9dc8-4379-8a60-0003ae931912 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:26.076921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 98/195] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:26.077741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97447ef2-9dc8-4379-8a60-0003ae931912 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:26.140896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:26.141208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4503e8d-8df7-48d4-bfa2-0d227dd3c305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:26.142829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4503e8d-8df7-48d4-bfa2-0d227dd3c305 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:26.173105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:26.193058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97447ef2-9dc8-4379-8a60-0003ae931912 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:26.225904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae9c36-5491-4332-b970-05c518d9d16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:26.226695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae9c36-5491-4332-b970-05c518d9d16e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:26.226977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae9c36-5491-4332-b970-05c518d9d16e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:26.227260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae9c36-5491-4332-b970-05c518d9d16e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:26.237238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae9c36-5491-4332-b970-05c518d9d16e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:26.242820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 98/196] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:26.243660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4503e8d-8df7-48d4-bfa2-0d227dd3c305 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6aae9c36-5491-4332-b970-05c518d9d16e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:26.245672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:26.246516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:26.247295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 99/197] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:26.247921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae9c36-5491-4332-b970-05c518d9d16e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:26.349012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae9c36-5491-4332-b970-05c518d9d16e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:26.349825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4503e8d-8df7-48d4-bfa2-0d227dd3c305 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:26.359083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:26.379108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:26.384063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d465a-8282-495b-9fcc-4f98dbd6a7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:26.384392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d965bc8-e649-49a8-a4fc-12e2f52a8aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:26.384745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d465a-8282-495b-9fcc-4f98dbd6a7d0 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:26.385027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d965bc8-e649-49a8-a4fc-12e2f52a8aa9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:26.385117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d465a-8282-495b-9fcc-4f98dbd6a7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:26.385303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d965bc8-e649-49a8-a4fc-12e2f52a8aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:26.385392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d465a-8282-495b-9fcc-4f98dbd6a7d0 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:26.385573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d965bc8-e649-49a8-a4fc-12e2f52a8aa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:26.386062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d465a-8282-495b-9fcc-4f98dbd6a7d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:26.387541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d465a-8282-495b-9fcc-4f98dbd6a7d0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:26.416097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 100/198] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:39:26 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:26.418704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d465a-8282-495b-9fcc-4f98dbd6a7d0 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:26.464687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d965bc8-e649-49a8-a4fc-12e2f52a8aa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:26.466253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d965bc8-e649-49a8-a4fc-12e2f52a8aa9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:26.538950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d465a-8282-495b-9fcc-4f98dbd6a7d0 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:26.584879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497d490-3dcc-4166-959b-189b20144be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:26.585809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d490-3dcc-4166-959b-189b20144be1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:26.586095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d490-3dcc-4166-959b-189b20144be1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:26.586967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d490-3dcc-4166-959b-189b20144be1 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:26.587087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 99/199] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:26.587945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d965bc8-e649-49a8-a4fc-12e2f52a8aa9 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:26.588309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497d490-3dcc-4166-959b-189b20144be1 None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:26.590035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497d490-3dcc-4166-959b-189b20144be1 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:26.591756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 101/200] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:26 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:26.594579 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497d490-3dcc-4166-959b-189b20144be1 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:26.619913 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:26.638502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:26.702463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d965bc8-e649-49a8-a4fc-12e2f52a8aa9 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:26.734237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497d490-3dcc-4166-959b-189b20144be1 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:26.747590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e2467-de43-49bf-9201-ff1d87bdf540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:26.748264 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e2467-de43-49bf-9201-ff1d87bdf540 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:26.748545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e2467-de43-49bf-9201-ff1d87bdf540 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:26.748870 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e2467-de43-49bf-9201-ff1d87bdf540 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:26.749628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e2467-de43-49bf-9201-ff1d87bdf540 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:26.749948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e2467-de43-49bf-9201-ff1d87bdf540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:26.751336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e2467-de43-49bf-9201-ff1d87bdf540 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:26.763391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 100/201] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:26.764637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e2467-de43-49bf-9201-ff1d87bdf540 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:26.793713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d44b62-a9e6-4ede-a35a-9a82f13e4139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:26.795096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d44b62-a9e6-4ede-a35a-9a82f13e4139 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:26.795506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d44b62-a9e6-4ede-a35a-9a82f13e4139 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:26.795913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d44b62-a9e6-4ede-a35a-9a82f13e4139 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:26.797469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d44b62-a9e6-4ede-a35a-9a82f13e4139 None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:26.799030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d44b62-a9e6-4ede-a35a-9a82f13e4139 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:26.800935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 102/202] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:26 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:26.801536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d44b62-a9e6-4ede-a35a-9a82f13e4139 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:26.891938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:26.902270 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e2467-de43-49bf-9201-ff1d87bdf540 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:26.937681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d44b62-a9e6-4ede-a35a-9a82f13e4139 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:26.938723 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0f30e-50b5-4623-ac2d-5e5f260f8e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:26.939207 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:26.940196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0f30e-50b5-4623-ac2d-5e5f260f8e47 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:26.940610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0f30e-50b5-4623-ac2d-5e5f260f8e47 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:26.941191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0f30e-50b5-4623-ac2d-5e5f260f8e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:26.942258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0f30e-50b5-4623-ac2d-5e5f260f8e47 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f, user.id=0d46c4c47bfb4ad0b2c58eca1669dcad) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:26.942708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0f30e-50b5-4623-ac2d-5e5f260f8e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:26.944297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0f30e-50b5-4623-ac2d-5e5f260f8e47 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:26.951285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 101/203] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:26 2022] GET /identity/v3/role_assignments?user.id=0d46c4c47bfb4ad0b2c58eca1669dcad&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:26.962803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0f30e-50b5-4623-ac2d-5e5f260f8e47 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:26.986677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f6190-f1d7-472e-a7b2-fcfcbb0a8b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:26.988399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f6190-f1d7-472e-a7b2-fcfcbb0a8b71 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:26.988683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f6190-f1d7-472e-a7b2-fcfcbb0a8b71 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:26.989015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f6190-f1d7-472e-a7b2-fcfcbb0a8b71 None admin] PATH_INFO: `/v3/users/c26dec6c07a148479302cc9f2e861a24` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:26.992233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f6190-f1d7-472e-a7b2-fcfcbb0a8b71 None admin] RBAC: Authorizing `identity:get_user(user_id=c26dec6c07a148479302cc9f2e861a24)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:26.993881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f6190-f1d7-472e-a7b2-fcfcbb0a8b71 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:26.995930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 103/204] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:26 2022] GET /identity/v3/users/c26dec6c07a148479302cc9f2e861a24 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:27.001425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f6190-f1d7-472e-a7b2-fcfcbb0a8b71 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:27.140525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0f30e-50b5-4623-ac2d-5e5f260f8e47 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:27.192189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc237e-9314-4920-861e-de010ad1a1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:27.192852 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc237e-9314-4920-861e-de010ad1a1d8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:27.193134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc237e-9314-4920-861e-de010ad1a1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:27.193412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc237e-9314-4920-861e-de010ad1a1d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:27.194286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc237e-9314-4920-861e-de010ad1a1d8 None admin] RBAC: Adding query filter params (role.id=b127031d393048d7ae9a35015113e82a, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f, user.id=0d46c4c47bfb4ad0b2c58eca1669dcad) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:27.194600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc237e-9314-4920-861e-de010ad1a1d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:27.196520 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc237e-9314-4920-861e-de010ad1a1d8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:27.202387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 102/205] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:26 2022] GET /identity/v3/role_assignments?user.id=0d46c4c47bfb4ad0b2c58eca1669dcad&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=b127031d393048d7ae9a35015113e82a => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:27.266046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f6190-f1d7-472e-a7b2-fcfcbb0a8b71 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:27.323514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d5a7a-0595-4a04-b4a4-9378425e4780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:27.324542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d5a7a-0595-4a04-b4a4-9378425e4780 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:27.325017 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d5a7a-0595-4a04-b4a4-9378425e4780 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:27.325482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d5a7a-0595-4a04-b4a4-9378425e4780 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:27.380793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:27.412352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d5a7a-0595-4a04-b4a4-9378425e4780 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:27.414209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d5a7a-0595-4a04-b4a4-9378425e4780 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:27.534710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 104/206] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:27.540026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc237e-9314-4920-861e-de010ad1a1d8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:27.580377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:27.664996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc237e-9314-4920-861e-de010ad1a1d8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:27.699856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0daf2f-bc13-4007-8cde-e7248a300c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:27.700712 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0daf2f-bc13-4007-8cde-e7248a300c2e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:27.700994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0daf2f-bc13-4007-8cde-e7248a300c2e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:27.701269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0daf2f-bc13-4007-8cde-e7248a300c2e None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:27.702495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0daf2f-bc13-4007-8cde-e7248a300c2e None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:27.703952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0daf2f-bc13-4007-8cde-e7248a300c2e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:27.706615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 103/207] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:27 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:27.712083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d5a7a-0595-4a04-b4a4-9378425e4780 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:27.829209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:27.869010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d5a7a-0595-4a04-b4a4-9378425e4780 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:27.909245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fea09-e320-4d56-be65-ff07875f22d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:27.909932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fea09-e320-4d56-be65-ff07875f22d3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:27.910217 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fea09-e320-4d56-be65-ff07875f22d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:27.910492 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fea09-e320-4d56-be65-ff07875f22d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:27.911591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fea09-e320-4d56-be65-ff07875f22d3 None admin] RBAC: Adding query filter params (role.id=0c5069e08baa4820b7154564a801ba7e, scope.project.id=574afa1590854c65a807df4589e2c460, user.id=c26dec6c07a148479302cc9f2e861a24) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:27.911899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fea09-e320-4d56-be65-ff07875f22d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:27.913345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fea09-e320-4d56-be65-ff07875f22d3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:27.919879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 105/208] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:27 2022] GET /identity/v3/role_assignments?user.id=c26dec6c07a148479302cc9f2e861a24&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=0c5069e08baa4820b7154564a801ba7e => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:28.730141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c98f82-f0c1-4241-ac22-cf77a21eed65 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:28.731146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c98f82-f0c1-4241-ac22-cf77a21eed65 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:28.731648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c98f82-f0c1-4241-ac22-cf77a21eed65 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:28.733487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 104/209] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:28.739087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e9950-4350-48f5-bed8-ebfbe3750422 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:28.739920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e9950-4350-48f5-bed8-ebfbe3750422 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:28.740415 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e9950-4350-48f5-bed8-ebfbe3750422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:28.759287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-728e9950-4350-48f5-bed8-ebfbe3750422 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:28.760864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-728e9950-4350-48f5-bed8-ebfbe3750422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:28.832745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 106/210] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:28.856209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c98f82-f0c1-4241-ac22-cf77a21eed65 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:28.954822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:28.965743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c98f82-f0c1-4241-ac22-cf77a21eed65 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:28.968007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f856962-1f3d-4f6f-9603-538e4ef57b35 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:28.968304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f856962-1f3d-4f6f-9603-538e4ef57b35 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:28.968584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f856962-1f3d-4f6f-9603-538e4ef57b35 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:28.970432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 107/211] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:28.975664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264685ac-d0a7-4d68-8b24-2490bc8f2c64 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:28.976214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264685ac-d0a7-4d68-8b24-2490bc8f2c64 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:28.976689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264685ac-d0a7-4d68-8b24-2490bc8f2c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:28.993936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-264685ac-d0a7-4d68-8b24-2490bc8f2c64 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:28.995387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-264685ac-d0a7-4d68-8b24-2490bc8f2c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:29.009919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786fe2c-d0e1-4558-9ddd-c369ec7c5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:29.011089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786fe2c-d0e1-4558-9ddd-c369ec7c5145 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.011371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786fe2c-d0e1-4558-9ddd-c369ec7c5145 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.011645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786fe2c-d0e1-4558-9ddd-c369ec7c5145 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.026069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786fe2c-d0e1-4558-9ddd-c369ec7c5145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-9786fe2c-d0e1-4558-9ddd-c369ec7c5145 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:29.035925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:29.037033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:29.037856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:29.045040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 105/212] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:29.046157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786fe2c-d0e1-4558-9ddd-c369ec7c5145 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:29.067963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:29.079287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 108/213] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:29.109954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264685ac-d0a7-4d68-8b24-2490bc8f2c64 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:29.161711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786fe2c-d0e1-4558-9ddd-c369ec7c5145 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:29.205605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96470720-4c84-4199-b7b0-66fcb246673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:29.205605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96470720-4c84-4199-b7b0-66fcb246673a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.205605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96470720-4c84-4199-b7b0-66fcb246673a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.205605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96470720-4c84-4199-b7b0-66fcb246673a None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.206028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96470720-4c84-4199-b7b0-66fcb246673a None admin] RBAC: Adding query filter params (name=member) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:29.206028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96470720-4c84-4199-b7b0-66fcb246673a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:29.206154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96470720-4c84-4199-b7b0-66fcb246673a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:29.215187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 106/214] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:29.222593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96470720-4c84-4199-b7b0-66fcb246673a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:29.233818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264685ac-d0a7-4d68-8b24-2490bc8f2c64 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:29.250092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:29.278397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e36373-33a8-45ab-901a-ec6e84a1f258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:29.280141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e36373-33a8-45ab-901a-ec6e84a1f258 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.280571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e36373-33a8-45ab-901a-ec6e84a1f258 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.281240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e36373-33a8-45ab-901a-ec6e84a1f258 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.282861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e36373-33a8-45ab-901a-ec6e84a1f258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:29.285636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e36373-33a8-45ab-901a-ec6e84a1f258 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:29.288284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 109/215] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:29.293937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e36373-33a8-45ab-901a-ec6e84a1f258 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:29.345453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96470720-4c84-4199-b7b0-66fcb246673a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:29.369354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:29.383770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8ad3e-2b6e-4c49-94fc-d69fcad61b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:29.385496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8ad3e-2b6e-4c49-94fc-d69fcad61b10 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.386018 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8ad3e-2b6e-4c49-94fc-d69fcad61b10 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.386468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8ad3e-2b6e-4c49-94fc-d69fcad61b10 None admin] PATH_INFO: `/v3/users/2c11d79ccdb5455693485ab60192fcdb` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.404000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8ad3e-2b6e-4c49-94fc-d69fcad61b10 None admin] RBAC: Authorizing `identity:get_user(user_id=2c11d79ccdb5455693485ab60192fcdb)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:29.405474 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8ad3e-2b6e-4c49-94fc-d69fcad61b10 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:29.407391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 107/216] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:29 2022] GET /identity/v3/users/2c11d79ccdb5455693485ab60192fcdb => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:29.408557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7bf0b-9363-461f-9dee-c4f8da5a78ba None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.409476 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7bf0b-9363-461f-9dee-c4f8da5a78ba None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.409773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7bf0b-9363-461f-9dee-c4f8da5a78ba None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.411027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 108/217] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:29.416045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7bf0b-9363-461f-9dee-c4f8da5a78ba None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:29.416658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e36373-33a8-45ab-901a-ec6e84a1f258 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:29.456439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f8c56-8d3a-4efc-a013-85f7eaf641d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:29.457478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f8c56-8d3a-4efc-a013-85f7eaf641d9 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.457916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f8c56-8d3a-4efc-a013-85f7eaf641d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.458338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f8c56-8d3a-4efc-a013-85f7eaf641d9 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.459217 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f8c56-8d3a-4efc-a013-85f7eaf641d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:29.461254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f8c56-8d3a-4efc-a013-85f7eaf641d9 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:29.488006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 110/218] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:39:29 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:39:29.489917 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c73a8-077f-4831-9213-8f3f88b80956 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.490346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c73a8-077f-4831-9213-8f3f88b80956 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.491007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c73a8-077f-4831-9213-8f3f88b80956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.499747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:29.515165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-845c73a8-077f-4831-9213-8f3f88b80956 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:29.517168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-845c73a8-077f-4831-9213-8f3f88b80956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:29.537853 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7bf0b-9363-461f-9dee-c4f8da5a78ba None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:29.587076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdda7e9-f1fb-4e3e-a070-03c4467051de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:29.587744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdda7e9-f1fb-4e3e-a070-03c4467051de None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.588294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdda7e9-f1fb-4e3e-a070-03c4467051de None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.588706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdda7e9-f1fb-4e3e-a070-03c4467051de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.623683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 111/219] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:29.649921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c73a8-077f-4831-9213-8f3f88b80956 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:29.685684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdda7e9-f1fb-4e3e-a070-03c4467051de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:29.686897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdda7e9-f1fb-4e3e-a070-03c4467051de None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:29.711764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:29.771454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c73a8-077f-4831-9213-8f3f88b80956 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:29.785676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:29.820846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fc99f-74ef-42bf-833d-948768775844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:29.821699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fc99f-74ef-42bf-833d-948768775844 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.821987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fc99f-74ef-42bf-833d-948768775844 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.822266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fc99f-74ef-42bf-833d-948768775844 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.832503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 109/220] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:29.834360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972fc99f-74ef-42bf-833d-948768775844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:29.834541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25fbe-4d54-4c82-8846-08c1b244f9d4 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.835016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25fbe-4d54-4c82-8846-08c1b244f9d4 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.835438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25fbe-4d54-4c82-8846-08c1b244f9d4 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.837179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 110/221] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:29.838105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b25fbe-4d54-4c82-8846-08c1b244f9d4 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-972fc99f-74ef-42bf-833d-948768775844 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:29.843411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:29.844457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:29.845417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:29.847056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 112/222] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:39:29.848131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008b28-003d-4fc4-80ef-e4ae3d975450 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:29.848418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008b28-003d-4fc4-80ef-e4ae3d975450 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:29.848694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008b28-003d-4fc4-80ef-e4ae3d975450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:29.873980 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-1e008b28-003d-4fc4-80ef-e4ae3d975450 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:29.875096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-1e008b28-003d-4fc4-80ef-e4ae3d975450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:29.956283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b25fbe-4d54-4c82-8846-08c1b244f9d4 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:29.965927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 113/223] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:29.968348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e008b28-003d-4fc4-80ef-e4ae3d975450 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:30.000168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc88f0-90d9-44f3-972b-d58453c6039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:30.002659 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc88f0-90d9-44f3-972b-d58453c6039f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:30.003123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc88f0-90d9-44f3-972b-d58453c6039f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:30.003752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc88f0-90d9-44f3-972b-d58453c6039f None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:30.005818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc88f0-90d9-44f3-972b-d58453c6039f None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:30.007466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc88f0-90d9-44f3-972b-d58453c6039f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:30.010324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 111/224] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:29 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:30.011294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc88f0-90d9-44f3-972b-d58453c6039f None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:30.045645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:30.062737 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:30.130252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e008b28-003d-4fc4-80ef-e4ae3d975450 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:30.141259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc88f0-90d9-44f3-972b-d58453c6039f None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:30.168603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fac48-ec57-4032-ae3f-75c13647f4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:30.169281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fac48-ec57-4032-ae3f-75c13647f4d4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:30.169563 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fac48-ec57-4032-ae3f-75c13647f4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:30.169866 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fac48-ec57-4032-ae3f-75c13647f4d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:30.170616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fac48-ec57-4032-ae3f-75c13647f4d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:30.170920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fac48-ec57-4032-ae3f-75c13647f4d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:30.172316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fac48-ec57-4032-ae3f-75c13647f4d4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:30.181920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 114/225] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:30.182748 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fac48-ec57-4032-ae3f-75c13647f4d4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:30.185386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064eb07-d20f-4388-a185-f29f2994b425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:30.186357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064eb07-d20f-4388-a185-f29f2994b425 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:30.186649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064eb07-d20f-4388-a185-f29f2994b425 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:30.186972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064eb07-d20f-4388-a185-f29f2994b425 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:30.198326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064eb07-d20f-4388-a185-f29f2994b425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-7064eb07-d20f-4388-a185-f29f2994b425 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:30.206784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:30.207662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:30.208440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 112/226] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:30.213568 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064eb07-d20f-4388-a185-f29f2994b425 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:30.311569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:30.317182 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fac48-ec57-4032-ae3f-75c13647f4d4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:30.319514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:30.323749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064eb07-d20f-4388-a185-f29f2994b425 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:30.356869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75899dc8-2237-4b94-a99a-5cb9127d8eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:30.357532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75899dc8-2237-4b94-a99a-5cb9127d8eb9 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:30.357844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75899dc8-2237-4b94-a99a-5cb9127d8eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:30.358122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75899dc8-2237-4b94-a99a-5cb9127d8eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:30.358961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75899dc8-2237-4b94-a99a-5cb9127d8eb9 None admin] RBAC: Adding query filter params (role.id=2e1db0cf27684f389b9320673721684d, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f, user.id=2c11d79ccdb5455693485ab60192fcdb) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:30.359271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75899dc8-2237-4b94-a99a-5cb9127d8eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:30.360331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d55fe-a297-4d5e-86ba-9055dee12179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:30.360717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75899dc8-2237-4b94-a99a-5cb9127d8eb9 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:30.361344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d55fe-a297-4d5e-86ba-9055dee12179 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:30.361753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d55fe-a297-4d5e-86ba-9055dee12179 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:30.362089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d55fe-a297-4d5e-86ba-9055dee12179 None admin] PATH_INFO: `/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:30.363382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d55fe-a297-4d5e-86ba-9055dee12179 None admin] RBAC: Authorizing `identity:get_user(user_id=0d46c4c47bfb4ad0b2c58eca1669dcad)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:30.365154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d55fe-a297-4d5e-86ba-9055dee12179 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:30.366745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 115/227] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:30 2022] GET /identity/v3/role_assignments?user.id=2c11d79ccdb5455693485ab60192fcdb&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=2e1db0cf27684f389b9320673721684d => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:30.367567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75899dc8-2237-4b94-a99a-5cb9127d8eb9 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:30.368149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 113/228] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:30 2022] GET /identity/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:30.378936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d55fe-a297-4d5e-86ba-9055dee12179 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:30.492612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75899dc8-2237-4b94-a99a-5cb9127d8eb9 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:30.535406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa58ad-0699-407f-afe2-5e6d82278006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:30.536092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa58ad-0699-407f-afe2-5e6d82278006 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:30.536628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa58ad-0699-407f-afe2-5e6d82278006 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:30.536897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa58ad-0699-407f-afe2-5e6d82278006 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:30.537709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa58ad-0699-407f-afe2-5e6d82278006 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:30.538012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa58ad-0699-407f-afe2-5e6d82278006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:30.539549 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa58ad-0699-407f-afe2-5e6d82278006 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:30.553038 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 116/229] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:30.560065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa58ad-0699-407f-afe2-5e6d82278006 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:30.578815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d55fe-a297-4d5e-86ba-9055dee12179 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:30.617627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:30.618005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451d2c4-390d-41ed-aae8-8f7339aca22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:30.618890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451d2c4-390d-41ed-aae8-8f7339aca22e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:30.619179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451d2c4-390d-41ed-aae8-8f7339aca22e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:30.619454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451d2c4-390d-41ed-aae8-8f7339aca22e None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:30.621402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451d2c4-390d-41ed-aae8-8f7339aca22e None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:30.622904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451d2c4-390d-41ed-aae8-8f7339aca22e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:30.625169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 114/230] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:30 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:30.631148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451d2c4-390d-41ed-aae8-8f7339aca22e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:30.684777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:30.733147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa58ad-0699-407f-afe2-5e6d82278006 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:30.759741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451d2c4-390d-41ed-aae8-8f7339aca22e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:30.796632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e491b-5093-417e-9ec0-816406bd5d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:30.798247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e491b-5093-417e-9ec0-816406bd5d29 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:30.798675 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e491b-5093-417e-9ec0-816406bd5d29 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:30.799121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e491b-5093-417e-9ec0-816406bd5d29 None admin] PATH_INFO: `/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:30.800973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e491b-5093-417e-9ec0-816406bd5d29 None admin] RBAC: Authorizing `identity:get_user(user_id=0d46c4c47bfb4ad0b2c58eca1669dcad)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:30.802677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e491b-5093-417e-9ec0-816406bd5d29 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:30.809233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 117/231] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:30 2022] GET /identity/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:30.810716 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e491b-5093-417e-9ec0-816406bd5d29 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:30.879676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e3542-0bdb-4e4d-b2fd-fbfce926300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:30.881139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e3542-0bdb-4e4d-b2fd-fbfce926300e None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:30.881602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e3542-0bdb-4e4d-b2fd-fbfce926300e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:30.882029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e3542-0bdb-4e4d-b2fd-fbfce926300e None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/users/0d46c4c47bfb4ad0b2c58eca1669dcad/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:30.884174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e3542-0bdb-4e4d-b2fd-fbfce926300e None admin] RBAC: Authorizing `identity:create_grant(project_id=3d0bca94154a400bb0d7c93638e9b20f, user_id=0d46c4c47bfb4ad0b2c58eca1669dcad, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:30.885789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e3542-0bdb-4e4d-b2fd-fbfce926300e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:30.921768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:30.932976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e491b-5093-417e-9ec0-816406bd5d29 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:30.937458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 115/232] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:39:30 2022] PUT /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/users/0d46c4c47bfb4ad0b2c58eca1669dcad/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:39:30.976174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb26f16-f22a-47e6-85e2-e72f5f14bd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:30.977109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb26f16-f22a-47e6-85e2-e72f5f14bd1d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:30.977394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb26f16-f22a-47e6-85e2-e72f5f14bd1d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:30.977731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb26f16-f22a-47e6-85e2-e72f5f14bd1d None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:30.979003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb26f16-f22a-47e6-85e2-e72f5f14bd1d None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:30.980588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb26f16-f22a-47e6-85e2-e72f5f14bd1d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:30.982644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 118/233] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:30 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:30.989289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e3542-0bdb-4e4d-b2fd-fbfce926300e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:31.092626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:31.102165 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e3542-0bdb-4e4d-b2fd-fbfce926300e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:31.144111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b51bcf-fab1-4e21-82b5-c44b5d0174db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:31.145036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b51bcf-fab1-4e21-82b5-c44b5d0174db None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:31.145319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b51bcf-fab1-4e21-82b5-c44b5d0174db None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:31.145607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b51bcf-fab1-4e21-82b5-c44b5d0174db None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/users/0d46c4c47bfb4ad0b2c58eca1669dcad/roles/b127031d393048d7ae9a35015113e82a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:31.151852 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b51bcf-fab1-4e21-82b5-c44b5d0174db None admin] RBAC: Authorizing `identity:create_grant(project_id=3d0bca94154a400bb0d7c93638e9b20f, user_id=0d46c4c47bfb4ad0b2c58eca1669dcad, role_id=b127031d393048d7ae9a35015113e82a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:31.153355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b51bcf-fab1-4e21-82b5-c44b5d0174db None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:31.189252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 116/234] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:39:30 2022] PUT /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/users/0d46c4c47bfb4ad0b2c58eca1669dcad/roles/b127031d393048d7ae9a35015113e82a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:31.946579 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e38f-f711-45b3-a60c-817fa41d9942 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:31.947082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e38f-f711-45b3-a60c-817fa41d9942 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:31.947155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e38f-f711-45b3-a60c-817fa41d9942 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:31.948714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 119/235] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:31.953981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38148a43-417c-4d7d-a8d1-bdf58e6c6ae7 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:31.954821 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38148a43-417c-4d7d-a8d1-bdf58e6c6ae7 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:31.955959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38148a43-417c-4d7d-a8d1-bdf58e6c6ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:31.981787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-38148a43-417c-4d7d-a8d1-bdf58e6c6ae7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:31.983002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-38148a43-417c-4d7d-a8d1-bdf58e6c6ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:32.073866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 117/236] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:32.105724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e38f-f711-45b3-a60c-817fa41d9942 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:32.148703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f2563-d0a8-466a-bbd7-f765c7089d1e None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:32.149016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f2563-d0a8-466a-bbd7-f765c7089d1e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:32.149288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f2563-d0a8-466a-bbd7-f765c7089d1e None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:32.150711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 118/237] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:39:32.157926 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff8d63-61cf-4fdc-be6e-aaaccc0e428c None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:32.158817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff8d63-61cf-4fdc-be6e-aaaccc0e428c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:32.159311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff8d63-61cf-4fdc-be6e-aaaccc0e428c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:32.203085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e4ff8d63-61cf-4fdc-be6e-aaaccc0e428c None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:32.205086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e4ff8d63-61cf-4fdc-be6e-aaaccc0e428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:32.248444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e38f-f711-45b3-a60c-817fa41d9942 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:32.294105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 119/238] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:32.316024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e71144-0464-429e-b63b-e998afa464e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:32.317121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e71144-0464-429e-b63b-e998afa464e7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:32.317615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e71144-0464-429e-b63b-e998afa464e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:32.318089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e71144-0464-429e-b63b-e998afa464e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:32.338394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e71144-0464-429e-b63b-e998afa464e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-e2e71144-0464-429e-b63b-e998afa464e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:32.350444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:32.351327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:32.352101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:32.354464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 120/239] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:32.358718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff8d63-61cf-4fdc-be6e-aaaccc0e428c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:32.363357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e71144-0464-429e-b63b-e998afa464e7 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:32.382874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:32.422955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:32.493635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff8d63-61cf-4fdc-be6e-aaaccc0e428c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:32.563980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcb25d-3ad5-422e-a95e-b1749dc89ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:32.564904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcb25d-3ad5-422e-a95e-b1749dc89ae2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:32.565508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcb25d-3ad5-422e-a95e-b1749dc89ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:32.565508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcb25d-3ad5-422e-a95e-b1749dc89ae2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:32.568857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e71144-0464-429e-b63b-e998afa464e7 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:32.571221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcb25d-3ad5-422e-a95e-b1749dc89ae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:32.572715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcb25d-3ad5-422e-a95e-b1749dc89ae2 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:32.574426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 120/240] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:32.582376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcb25d-3ad5-422e-a95e-b1749dc89ae2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:32.633490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953e6f3-ce2c-430b-9ff9-759c1e8351ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:32.634190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953e6f3-ce2c-430b-9ff9-759c1e8351ee None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:32.634472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953e6f3-ce2c-430b-9ff9-759c1e8351ee None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:32.634744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953e6f3-ce2c-430b-9ff9-759c1e8351ee None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:32.635499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953e6f3-ce2c-430b-9ff9-759c1e8351ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:32.635802 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953e6f3-ce2c-430b-9ff9-759c1e8351ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:32.637191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953e6f3-ce2c-430b-9ff9-759c1e8351ee None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:32.662153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 121/241] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:32.667657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953e6f3-ce2c-430b-9ff9-759c1e8351ee None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:32.740382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:32.761451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:32.772297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcb25d-3ad5-422e-a95e-b1749dc89ae2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:32.780349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953e6f3-ce2c-430b-9ff9-759c1e8351ee None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:32.818109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f930043-5cb9-416a-bd5d-4f823f9544eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:32.820054 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f930043-5cb9-416a-bd5d-4f823f9544eb None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:32.823046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f930043-5cb9-416a-bd5d-4f823f9544eb None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:32.823475 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f930043-5cb9-416a-bd5d-4f823f9544eb None admin] PATH_INFO: `/v3/users/143d447710be462788b05d34fabb23f9` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:32.849728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f930043-5cb9-416a-bd5d-4f823f9544eb None admin] RBAC: Authorizing `identity:get_user(user_id=143d447710be462788b05d34fabb23f9)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:32.852324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f930043-5cb9-416a-bd5d-4f823f9544eb None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:32.853078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606cfa2-e589-4101-aaef-c216d20a670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:32.853928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606cfa2-e589-4101-aaef-c216d20a670c None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:32.854346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606cfa2-e589-4101-aaef-c216d20a670c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:32.854777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606cfa2-e589-4101-aaef-c216d20a670c None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:32.856638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 122/242] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:32 2022] GET /identity/v3/users/143d447710be462788b05d34fabb23f9 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:32.857668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606cfa2-e589-4101-aaef-c216d20a670c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:32.859266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606cfa2-e589-4101-aaef-c216d20a670c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:32.866963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f930043-5cb9-416a-bd5d-4f823f9544eb None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:32.903694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 121/243] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:39:32 2022] POST /identity/v3/users => generated 293 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:32.983605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f930043-5cb9-416a-bd5d-4f823f9544eb None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:33.022681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc9583-5688-459e-b26b-be83a5a1ae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:33.023587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc9583-5688-459e-b26b-be83a5a1ae6c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.024022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc9583-5688-459e-b26b-be83a5a1ae6c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.024457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc9583-5688-459e-b26b-be83a5a1ae6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.038854 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:33.068415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5af034-0d59-4854-8ef8-52b94e007681 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.069266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5af034-0d59-4854-8ef8-52b94e007681 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.069736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5af034-0d59-4854-8ef8-52b94e007681 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.071472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 122/244] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:39:33.077284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86d99b-2e93-4fc5-a3c9-44ca6333b4b2 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.077877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86d99b-2e93-4fc5-a3c9-44ca6333b4b2 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.078361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86d99b-2e93-4fc5-a3c9-44ca6333b4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.101513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc9583-5688-459e-b26b-be83a5a1ae6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:33.103020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc9583-5688-459e-b26b-be83a5a1ae6c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:33.112901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-bd86d99b-2e93-4fc5-a3c9-44ca6333b4b2 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:33.114430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-bd86d99b-2e93-4fc5-a3c9-44ca6333b4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:33.217129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 123/245] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:33.227407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 123/246] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:33.232686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86d99b-2e93-4fc5-a3c9-44ca6333b4b2 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:33.245451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc9583-5688-459e-b26b-be83a5a1ae6c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:33.309764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:33.335484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:33.384773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86d99b-2e93-4fc5-a3c9-44ca6333b4b2 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:33.386910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc9583-5688-459e-b26b-be83a5a1ae6c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:33.433976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf9ca3-73db-47e1-94ee-aeca566c2f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:33.434844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf9ca3-73db-47e1-94ee-aeca566c2f1b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.435128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf9ca3-73db-47e1-94ee-aeca566c2f1b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.435405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf9ca3-73db-47e1-94ee-aeca566c2f1b None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.436906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf9ca3-73db-47e1-94ee-aeca566c2f1b None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:33.438468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf9ca3-73db-47e1-94ee-aeca566c2f1b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:33.439137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54247e-b146-461d-910c-63f0c663b147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:33.440356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54247e-b146-461d-910c-63f0c663b147 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.440669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 124/247] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:33 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:33.440947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54247e-b146-461d-910c-63f0c663b147 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.441288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54247e-b146-461d-910c-63f0c663b147 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.446058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf9ca3-73db-47e1-94ee-aeca566c2f1b None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:33.453369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54247e-b146-461d-910c-63f0c663b147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-4d54247e-b146-461d-910c-63f0c663b147 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:33.463118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:33.464083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:33.464848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 124/248] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:33.478650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54247e-b146-461d-910c-63f0c663b147 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:33.589020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf9ca3-73db-47e1-94ee-aeca566c2f1b None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:33.641079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54247e-b146-461d-910c-63f0c663b147 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:33.644172 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:33.660346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f1ebf-70b6-4655-97ee-6780e18e01fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:33.661683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f1ebf-70b6-4655-97ee-6780e18e01fe None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.662170 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f1ebf-70b6-4655-97ee-6780e18e01fe None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.663915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f1ebf-70b6-4655-97ee-6780e18e01fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.664980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f1ebf-70b6-4655-97ee-6780e18e01fe None admin] RBAC: Adding query filter params (role.id=0c5069e08baa4820b7154564a801ba7e, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f, user.id=143d447710be462788b05d34fabb23f9) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:33.666204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f1ebf-70b6-4655-97ee-6780e18e01fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:33.668114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f1ebf-70b6-4655-97ee-6780e18e01fe None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:33.675872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:33.682930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 125/249] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:33 2022] GET /identity/v3/role_assignments?user.id=143d447710be462788b05d34fabb23f9&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=0c5069e08baa4820b7154564a801ba7e => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:33.699040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8ff6b-39f5-4e97-9052-30e2445be30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:33.700074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8ff6b-39f5-4e97-9052-30e2445be30f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.700587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8ff6b-39f5-4e97-9052-30e2445be30f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.701075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8ff6b-39f5-4e97-9052-30e2445be30f None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.702154 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8ff6b-39f5-4e97-9052-30e2445be30f None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:33.702660 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8ff6b-39f5-4e97-9052-30e2445be30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:33.704293 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8ff6b-39f5-4e97-9052-30e2445be30f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:33.714793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 125/250] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:33.720484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f1ebf-70b6-4655-97ee-6780e18e01fe None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:33.739315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2eb7d1-21cc-4906-a844-91e31d91a7bf None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.739924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2eb7d1-21cc-4906-a844-91e31d91a7bf None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.740402 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2eb7d1-21cc-4906-a844-91e31d91a7bf None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.742184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 126/251] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:39:33.751376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe374e-c9d0-45f0-ab56-02e8c15cf0a0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.752557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe374e-c9d0-45f0-ab56-02e8c15cf0a0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.753035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe374e-c9d0-45f0-ab56-02e8c15cf0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.786526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-cbfe374e-c9d0-45f0-ab56-02e8c15cf0a0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:33.788136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-cbfe374e-c9d0-45f0-ab56-02e8c15cf0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:33.872109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f1ebf-70b6-4655-97ee-6780e18e01fe None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:33.912636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 127/252] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:33.914432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518d18c-3f8f-4a9a-80dc-4e79d0130053 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.914940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518d18c-3f8f-4a9a-80dc-4e79d0130053 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.915422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518d18c-3f8f-4a9a-80dc-4e79d0130053 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.916913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 128/253] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:33.920473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ae78d-2422-45f4-8e9b-88d8d66ae190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:33.922574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9169074-9f66-4c25-a7a6-41cea9cc38e5 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.923061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9169074-9f66-4c25-a7a6-41cea9cc38e5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.923553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9169074-9f66-4c25-a7a6-41cea9cc38e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.926094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ae78d-2422-45f4-8e9b-88d8d66ae190 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:33.926966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ae78d-2422-45f4-8e9b-88d8d66ae190 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:33.927980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ae78d-2422-45f4-8e9b-88d8d66ae190 None admin] PATH_INFO: `/v3/users/2c11d79ccdb5455693485ab60192fcdb` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:33.930576 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ae78d-2422-45f4-8e9b-88d8d66ae190 None admin] RBAC: Authorizing `identity:get_user(user_id=2c11d79ccdb5455693485ab60192fcdb)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:33.933918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ae78d-2422-45f4-8e9b-88d8d66ae190 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:33.937507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 126/254] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:33 2022] GET /identity/v3/users/2c11d79ccdb5455693485ab60192fcdb => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:33.939256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ae78d-2422-45f4-8e9b-88d8d66ae190 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:33.949100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a9169074-9f66-4c25-a7a6-41cea9cc38e5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:33.950743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a9169074-9f66-4c25-a7a6-41cea9cc38e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:34.014933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:34.060497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ae78d-2422-45f4-8e9b-88d8d66ae190 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:34.074780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 129/255] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:34.076228 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9169074-9f66-4c25-a7a6-41cea9cc38e5 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:34.103304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:34.129229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41288d6-443d-4bd3-a3ea-46c1c1a89fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:34.130624 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41288d6-443d-4bd3-a3ea-46c1c1a89fed None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:34.131051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41288d6-443d-4bd3-a3ea-46c1c1a89fed None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:34.131504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41288d6-443d-4bd3-a3ea-46c1c1a89fed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:34.150976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41288d6-443d-4bd3-a3ea-46c1c1a89fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-a41288d6-443d-4bd3-a3ea-46c1c1a89fed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:34.168638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:34.169545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:34.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:34.172480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 127/256] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:34.173467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41288d6-443d-4bd3-a3ea-46c1c1a89fed None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:34.202192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9169074-9f66-4c25-a7a6-41cea9cc38e5 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:34.282010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2787f-f83b-4fe4-8273-1a7a12d9b6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:34.283838 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2787f-f83b-4fe4-8273-1a7a12d9b6a5 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:34.284548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2787f-f83b-4fe4-8273-1a7a12d9b6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:34.284971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2787f-f83b-4fe4-8273-1a7a12d9b6a5 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:34.288440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2787f-f83b-4fe4-8273-1a7a12d9b6a5 None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:34.293078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2787f-f83b-4fe4-8273-1a7a12d9b6a5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:34.295110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 130/257] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:34 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:34.296346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2787f-f83b-4fe4-8273-1a7a12d9b6a5 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:34.336057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41288d6-443d-4bd3-a3ea-46c1c1a89fed None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:34.367507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:34.378220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139994-4a2d-470e-8e49-060e677fcc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:34.379909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139994-4a2d-470e-8e49-060e677fcc66 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:34.380400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139994-4a2d-470e-8e49-060e677fcc66 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:34.380927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf139994-4a2d-470e-8e49-060e677fcc66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:34.404567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf139994-4a2d-470e-8e49-060e677fcc66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-cf139994-4a2d-470e-8e49-060e677fcc66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:34.419259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:34.420187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 128/258] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:34.426011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139994-4a2d-470e-8e49-060e677fcc66 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:34.457390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:34.467036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2787f-f83b-4fe4-8273-1a7a12d9b6a5 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:34.516500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedafa4d-8d31-4c33-b217-8a7c057fceaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:34.517692 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedafa4d-8d31-4c33-b217-8a7c057fceaa None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:34.518202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedafa4d-8d31-4c33-b217-8a7c057fceaa None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:34.518689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedafa4d-8d31-4c33-b217-8a7c057fceaa None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:34.519641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedafa4d-8d31-4c33-b217-8a7c057fceaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:34.520234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedafa4d-8d31-4c33-b217-8a7c057fceaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:34.521972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedafa4d-8d31-4c33-b217-8a7c057fceaa None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:34.546284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 131/259] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:34.547451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedafa4d-8d31-4c33-b217-8a7c057fceaa None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:34.574134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf139994-4a2d-470e-8e49-060e677fcc66 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:34.615629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23371b-4619-4f31-aa19-d2d7f36a6021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:34.615629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23371b-4619-4f31-aa19-d2d7f36a6021 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:34.615629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23371b-4619-4f31-aa19-d2d7f36a6021 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:34.615629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23371b-4619-4f31-aa19-d2d7f36a6021 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/users/2c11d79ccdb5455693485ab60192fcdb/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:34.615992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23371b-4619-4f31-aa19-d2d7f36a6021 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d0bca94154a400bb0d7c93638e9b20f, user_id=2c11d79ccdb5455693485ab60192fcdb, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:34.616090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23371b-4619-4f31-aa19-d2d7f36a6021 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:34.636284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 129/260] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:39:34 2022] PUT /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/users/2c11d79ccdb5455693485ab60192fcdb/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:34.637152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23371b-4619-4f31-aa19-d2d7f36a6021 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:34.745310 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedafa4d-8d31-4c33-b217-8a7c057fceaa None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:34.758083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:34.839710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccfc02-cfb1-4f9b-9885-a587a4f3e598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:34.840454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccfc02-cfb1-4f9b-9885-a587a4f3e598 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:34.840742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccfc02-cfb1-4f9b-9885-a587a4f3e598 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:34.841023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccfc02-cfb1-4f9b-9885-a587a4f3e598 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:34.841817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccfc02-cfb1-4f9b-9885-a587a4f3e598 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:34.842132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccfc02-cfb1-4f9b-9885-a587a4f3e598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:34.843883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccfc02-cfb1-4f9b-9885-a587a4f3e598 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:34.861548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 132/261] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:34.868070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccfc02-cfb1-4f9b-9885-a587a4f3e598 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:34.876122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23371b-4619-4f31-aa19-d2d7f36a6021 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:34.910487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:35.024289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e70c9-08ba-44c9-9633-bedbbfef2cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:35.025193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e70c9-08ba-44c9-9633-bedbbfef2cb4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:35.025675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e70c9-08ba-44c9-9633-bedbbfef2cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:35.025952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e70c9-08ba-44c9-9633-bedbbfef2cb4 None admin] PATH_INFO: `/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:35.027469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e70c9-08ba-44c9-9633-bedbbfef2cb4 None admin] RBAC: Authorizing `identity:get_user(user_id=0d46c4c47bfb4ad0b2c58eca1669dcad)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:35.029104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e70c9-08ba-44c9-9633-bedbbfef2cb4 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:35.030948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 130/262] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:34 2022] GET /identity/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad => generated 283 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:35.038036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccfc02-cfb1-4f9b-9885-a587a4f3e598 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:35.038760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e70c9-08ba-44c9-9633-bedbbfef2cb4 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:35.080141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303665f4-39c5-408b-b76d-94ff4920fe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:35.081057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303665f4-39c5-408b-b76d-94ff4920fe54 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:35.081557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303665f4-39c5-408b-b76d-94ff4920fe54 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:35.081903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303665f4-39c5-408b-b76d-94ff4920fe54 None admin] PATH_INFO: `/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:35.083354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303665f4-39c5-408b-b76d-94ff4920fe54 None admin] RBAC: Authorizing `identity:get_user(user_id=0d46c4c47bfb4ad0b2c58eca1669dcad)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:35.085238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303665f4-39c5-408b-b76d-94ff4920fe54 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:35.091983 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 133/263] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:34 2022] GET /identity/v3/users/0d46c4c47bfb4ad0b2c58eca1669dcad => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:35.097388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303665f4-39c5-408b-b76d-94ff4920fe54 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:35.249779 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:35.275597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303665f4-39c5-408b-b76d-94ff4920fe54 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:35.294487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e70c9-08ba-44c9-9633-bedbbfef2cb4 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:35.327010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:35.331076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acb18e-3f50-4665-8cc6-6a451cb60893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:35.332596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acb18e-3f50-4665-8cc6-6a451cb60893 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:35.332882 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acb18e-3f50-4665-8cc6-6a451cb60893 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:35.333158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acb18e-3f50-4665-8cc6-6a451cb60893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:35.376739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21818-135c-4e21-a56b-062cdcf2a08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:35.378090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21818-135c-4e21-a56b-062cdcf2a08e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:35.378376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21818-135c-4e21-a56b-062cdcf2a08e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:35.378652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21818-135c-4e21-a56b-062cdcf2a08e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:35.429769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acb18e-3f50-4665-8cc6-6a451cb60893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:35.431260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acb18e-3f50-4665-8cc6-6a451cb60893 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:35.546768 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b21818-135c-4e21-a56b-062cdcf2a08e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:35.548286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b21818-135c-4e21-a56b-062cdcf2a08e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:35.579462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:35.612405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 134/264] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:35.617821 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acb18e-3f50-4665-8cc6-6a451cb60893 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:35.804822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 131/265] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 766 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:35.809851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21818-135c-4e21-a56b-062cdcf2a08e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:35.823718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acb18e-3f50-4665-8cc6-6a451cb60893 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:35.850875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:35.894236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e96ca1-55f6-43aa-92fe-3cb9ae170862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:35.895218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e96ca1-55f6-43aa-92fe-3cb9ae170862 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:35.895571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e96ca1-55f6-43aa-92fe-3cb9ae170862 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:35.895916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e96ca1-55f6-43aa-92fe-3cb9ae170862 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:35.897193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e96ca1-55f6-43aa-92fe-3cb9ae170862 None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:35.900557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e96ca1-55f6-43aa-92fe-3cb9ae170862 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:35.902587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 135/266] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:35 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:35.912229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e96ca1-55f6-43aa-92fe-3cb9ae170862 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:35.995357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21818-135c-4e21-a56b-062cdcf2a08e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:36.034860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:36.039238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc8613-aedf-45bb-b1be-b865c40cc703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:36.040573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc8613-aedf-45bb-b1be-b865c40cc703 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.041000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc8613-aedf-45bb-b1be-b865c40cc703 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.041420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc8613-aedf-45bb-b1be-b865c40cc703 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.045787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc8613-aedf-45bb-b1be-b865c40cc703 None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:36.047292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc8613-aedf-45bb-b1be-b865c40cc703 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:36.049694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 132/267] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:35 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:36.050939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b786c-7ef0-435f-8fcd-c869dfa430e4 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.051218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b786c-7ef0-435f-8fcd-c869dfa430e4 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.051495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b786c-7ef0-435f-8fcd-c869dfa430e4 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.052913 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 133/268] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:36.067553 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ebeb-691c-4bf6-9d4f-0431bf8527cd None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.067906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ebeb-691c-4bf6-9d4f-0431bf8527cd None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.068105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ebeb-691c-4bf6-9d4f-0431bf8527cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.097425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-8bf0ebeb-691c-4bf6-9d4f-0431bf8527cd None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:36.098652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8bf0ebeb-691c-4bf6-9d4f-0431bf8527cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:36.142179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e96ca1-55f6-43aa-92fe-3cb9ae170862 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:36.215884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dff1e-81bb-4da7-bee0-0cc2a7be419e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:36.216552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dff1e-81bb-4da7-bee0-0cc2a7be419e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.216833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dff1e-81bb-4da7-bee0-0cc2a7be419e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.217109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dff1e-81bb-4da7-bee0-0cc2a7be419e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.217991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dff1e-81bb-4da7-bee0-0cc2a7be419e None admin] RBAC: Adding query filter params (role.id=b127031d393048d7ae9a35015113e82a, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f, user.id=0d46c4c47bfb4ad0b2c58eca1669dcad) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:36.218306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dff1e-81bb-4da7-bee0-0cc2a7be419e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:36.219757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dff1e-81bb-4da7-bee0-0cc2a7be419e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:36.223820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 134/269] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:36.224689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0ebeb-691c-4bf6-9d4f-0431bf8527cd None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:36.241431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 136/270] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:35 2022] GET /identity/v3/role_assignments?user.id=0d46c4c47bfb4ad0b2c58eca1669dcad&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=b127031d393048d7ae9a35015113e82a => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:36.265058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:36.307894 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e3f7b-c1f8-4031-93f9-415ecfd4f23f None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.308216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e3f7b-c1f8-4031-93f9-415ecfd4f23f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.308458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e3f7b-c1f8-4031-93f9-415ecfd4f23f None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.310185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 137/271] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:36.314254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e3f7b-c1f8-4031-93f9-415ecfd4f23f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:36.368465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0ebeb-691c-4bf6-9d4f-0431bf8527cd None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:36.411797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a979ce3-df78-400e-bc40-375336126947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:36.412472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a979ce3-df78-400e-bc40-375336126947 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.412754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a979ce3-df78-400e-bc40-375336126947 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.413022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a979ce3-df78-400e-bc40-375336126947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.413906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a979ce3-df78-400e-bc40-375336126947 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f, user.id=0d46c4c47bfb4ad0b2c58eca1669dcad) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:36.414214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a979ce3-df78-400e-bc40-375336126947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:36.415719 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a979ce3-df78-400e-bc40-375336126947 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:36.434601 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 135/272] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:36 2022] GET /identity/v3/role_assignments?user.id=0d46c4c47bfb4ad0b2c58eca1669dcad&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:36.437617 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d04e3-5fce-4a40-98c8-8597c0670626 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.438069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d04e3-5fce-4a40-98c8-8597c0670626 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.439183 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d04e3-5fce-4a40-98c8-8597c0670626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.460734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-6c6d04e3-5fce-4a40-98c8-8597c0670626 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:36.462224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6c6d04e3-5fce-4a40-98c8-8597c0670626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:36.526654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e3f7b-c1f8-4031-93f9-415ecfd4f23f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:36.560663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:36.566630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 136/273] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:36.567988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b566ac-24e2-41a4-a18b-b3a888da587a None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.568822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b566ac-24e2-41a4-a18b-b3a888da587a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.569171 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b566ac-24e2-41a4-a18b-b3a888da587a None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.571567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 137/274] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:36.577927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729c1ad-93ab-4a60-af71-807abfa5bcc6 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.578208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729c1ad-93ab-4a60-af71-807abfa5bcc6 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.578809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729c1ad-93ab-4a60-af71-807abfa5bcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.591804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2cc3a-8959-4c99-ac8d-0eed98d2aa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:36.593645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2cc3a-8959-4c99-ac8d-0eed98d2aa52 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.594134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2cc3a-8959-4c99-ac8d-0eed98d2aa52 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.594906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2cc3a-8959-4c99-ac8d-0eed98d2aa52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.596668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2cc3a-8959-4c99-ac8d-0eed98d2aa52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:36.598466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2cc3a-8959-4c99-ac8d-0eed98d2aa52 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:36.598755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7729c1ad-93ab-4a60-af71-807abfa5bcc6 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:36.600027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7729c1ad-93ab-4a60-af71-807abfa5bcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:36.600787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 138/275] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:36.605804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:36.607033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2cc3a-8959-4c99-ac8d-0eed98d2aa52 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:36.694941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 138/276] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:36.695910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729c1ad-93ab-4a60-af71-807abfa5bcc6 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:36.747066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2cc3a-8959-4c99-ac8d-0eed98d2aa52 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:36.788294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0ae1b-c05b-4dc9-89e8-e36fe089de55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:36.788963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0ae1b-c05b-4dc9-89e8-e36fe089de55 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.789234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0ae1b-c05b-4dc9-89e8-e36fe089de55 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.789709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0ae1b-c05b-4dc9-89e8-e36fe089de55 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.790225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0ae1b-c05b-4dc9-89e8-e36fe089de55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:36.791663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0ae1b-c05b-4dc9-89e8-e36fe089de55 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:36.815544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729c1ad-93ab-4a60-af71-807abfa5bcc6 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:36.816145 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 139/277] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:39:36 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:36.817048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0ae1b-c05b-4dc9-89e8-e36fe089de55 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:36.864961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:36.871488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e08b0-93e6-4ec8-8eda-5c681182fa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:36.872793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e08b0-93e6-4ec8-8eda-5c681182fa79 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.873351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e08b0-93e6-4ec8-8eda-5c681182fa79 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.873814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e08b0-93e6-4ec8-8eda-5c681182fa79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.884414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:36.892196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e08b0-93e6-4ec8-8eda-5c681182fa79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-e06e08b0-93e6-4ec8-8eda-5c681182fa79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:36.904526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:36.905749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:36.906743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:36.909478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 139/278] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:36.916000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e08b0-93e6-4ec8-8eda-5c681182fa79 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:36.944784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0ae1b-c05b-4dc9-89e8-e36fe089de55 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:36.985323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d41f0-aa7b-41ba-b07f-b63ddc17b6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:36.985997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d41f0-aa7b-41ba-b07f-b63ddc17b6c0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:36.986909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d41f0-aa7b-41ba-b07f-b63ddc17b6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:36.987243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d41f0-aa7b-41ba-b07f-b63ddc17b6c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:36.999424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d41f0-aa7b-41ba-b07f-b63ddc17b6c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-d90d41f0-aa7b-41ba-b07f-b63ddc17b6c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:37.007932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:37.008820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:37.009611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 140/279] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:37.018568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d41f0-aa7b-41ba-b07f-b63ddc17b6c0 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:37.037959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e08b0-93e6-4ec8-8eda-5c681182fa79 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:37.076745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d589ac-6807-4130-8e87-de3f2277b26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:37.077420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d589ac-6807-4130-8e87-de3f2277b26a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:37.077733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d589ac-6807-4130-8e87-de3f2277b26a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:37.078011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d589ac-6807-4130-8e87-de3f2277b26a None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:37.078762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d589ac-6807-4130-8e87-de3f2277b26a None admin] RBAC: Adding query filter params (name=member) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:37.079083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d589ac-6807-4130-8e87-de3f2277b26a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:37.081119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d589ac-6807-4130-8e87-de3f2277b26a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:37.089722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 140/280] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:37.098559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d589ac-6807-4130-8e87-de3f2277b26a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:37.147306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:37.150677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d41f0-aa7b-41ba-b07f-b63ddc17b6c0 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:37.179184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:37.203456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e73e8a-02c0-42ab-8032-51d344a7b815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:37.203456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e73e8a-02c0-42ab-8032-51d344a7b815 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:37.203456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e73e8a-02c0-42ab-8032-51d344a7b815 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:37.203456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e73e8a-02c0-42ab-8032-51d344a7b815 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:37.204062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e73e8a-02c0-42ab-8032-51d344a7b815 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:37.204371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e73e8a-02c0-42ab-8032-51d344a7b815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:37.206696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e73e8a-02c0-42ab-8032-51d344a7b815 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:37.217284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 141/281] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:37.223649 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e73e8a-02c0-42ab-8032-51d344a7b815 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:37.233930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d589ac-6807-4130-8e87-de3f2277b26a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:37.277221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218dda4-6391-4000-a613-f6781330cdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:37.278033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218dda4-6391-4000-a613-f6781330cdb8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:37.278317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218dda4-6391-4000-a613-f6781330cdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:37.278590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218dda4-6391-4000-a613-f6781330cdb8 None admin] PATH_INFO: `/v3/users/33104eb5a28b421984dbea9fa0bb91d7` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:37.299070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218dda4-6391-4000-a613-f6781330cdb8 None admin] RBAC: Authorizing `identity:get_user(user_id=33104eb5a28b421984dbea9fa0bb91d7)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:37.301833 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218dda4-6391-4000-a613-f6781330cdb8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:37.303772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 141/282] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:37 2022] GET /identity/v3/users/33104eb5a28b421984dbea9fa0bb91d7 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:37.309827 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218dda4-6391-4000-a613-f6781330cdb8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:37.349414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e73e8a-02c0-42ab-8032-51d344a7b815 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:37.389197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337cb16-3022-4306-a9fe-ab1f052e91d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:37.390116 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337cb16-3022-4306-a9fe-ab1f052e91d7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:37.390394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337cb16-3022-4306-a9fe-ab1f052e91d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:37.390671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337cb16-3022-4306-a9fe-ab1f052e91d7 None admin] PATH_INFO: `/v3/users/143d447710be462788b05d34fabb23f9` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:37.391878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337cb16-3022-4306-a9fe-ab1f052e91d7 None admin] RBAC: Authorizing `identity:get_user(user_id=143d447710be462788b05d34fabb23f9)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:37.393920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337cb16-3022-4306-a9fe-ab1f052e91d7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:37.395905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 142/283] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:37 2022] GET /identity/v3/users/143d447710be462788b05d34fabb23f9 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:37.402029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337cb16-3022-4306-a9fe-ab1f052e91d7 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:37.439785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218dda4-6391-4000-a613-f6781330cdb8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:37.464367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:37.480235 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:37.509631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3774f62-c1bc-4c78-91e6-8746868315e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:37.510656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3774f62-c1bc-4c78-91e6-8746868315e9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:37.511137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3774f62-c1bc-4c78-91e6-8746868315e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:37.511646 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3774f62-c1bc-4c78-91e6-8746868315e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:37.512747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3774f62-c1bc-4c78-91e6-8746868315e9 None admin] RBAC: Adding query filter params (role.id=2e1db0cf27684f389b9320673721684d, user.id=33104eb5a28b421984dbea9fa0bb91d7, scope.system=all) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:37.513265 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3774f62-c1bc-4c78-91e6-8746868315e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:37.514950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3774f62-c1bc-4c78-91e6-8746868315e9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:37.521356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 142/284] 10.222.0.66 () {56 vars in 1308 bytes} [Fri Jul 1 19:39:37 2022] GET /identity/v3/role_assignments?user.id=33104eb5a28b421984dbea9fa0bb91d7&scope.system=all&role.id=2e1db0cf27684f389b9320673721684d => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:37.607808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337cb16-3022-4306-a9fe-ab1f052e91d7 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:37.656295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eedcda-1f28-4e46-bec7-e70a08d4842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:37.657194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eedcda-1f28-4e46-bec7-e70a08d4842c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:37.657472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eedcda-1f28-4e46-bec7-e70a08d4842c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:37.657769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eedcda-1f28-4e46-bec7-e70a08d4842c None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:37.662404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eedcda-1f28-4e46-bec7-e70a08d4842c None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:37.663907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eedcda-1f28-4e46-bec7-e70a08d4842c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:37.665955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 143/285] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:37 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:37.672019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3774f62-c1bc-4c78-91e6-8746868315e9 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:37.813513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3774f62-c1bc-4c78-91e6-8746868315e9 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:37.857206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7069f-b378-4f2c-bd53-b45c6f719a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:37.858136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7069f-b378-4f2c-bd53-b45c6f719a54 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:37.858440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7069f-b378-4f2c-bd53-b45c6f719a54 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:37.858705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7069f-b378-4f2c-bd53-b45c6f719a54 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/users/143d447710be462788b05d34fabb23f9/roles/0c5069e08baa4820b7154564a801ba7e` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:37.860751 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7069f-b378-4f2c-bd53-b45c6f719a54 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d0bca94154a400bb0d7c93638e9b20f, user_id=143d447710be462788b05d34fabb23f9, role_id=0c5069e08baa4820b7154564a801ba7e)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:37.862931 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7069f-b378-4f2c-bd53-b45c6f719a54 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:37.866561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d9605-24c6-4e7e-98e9-191b4ef99a03 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:37.866845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d9605-24c6-4e7e-98e9-191b4ef99a03 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:37.867268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d9605-24c6-4e7e-98e9-191b4ef99a03 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:37.868804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 144/286] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:37.874093 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b5ef6-c6c2-4b7d-9d59-13c27cbb87c5 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:37.874926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b5ef6-c6c2-4b7d-9d59-13c27cbb87c5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:37.875418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b5ef6-c6c2-4b7d-9d59-13c27cbb87c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:37.884987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 143/287] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:39:37 2022] PUT /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/users/143d447710be462788b05d34fabb23f9/roles/0c5069e08baa4820b7154564a801ba7e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:37.897288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-889b5ef6-c6c2-4b7d-9d59-13c27cbb87c5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:37.898861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-889b5ef6-c6c2-4b7d-9d59-13c27cbb87c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:37.996005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 145/288] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:38.038588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7069f-b378-4f2c-bd53-b45c6f719a54 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:38.098368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:38.163941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7069f-b378-4f2c-bd53-b45c6f719a54 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:38.201291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7990c71-6317-4cc1-b0a3-a061c0c9327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:38.202376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7990c71-6317-4cc1-b0a3-a061c0c9327d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:38.202803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7990c71-6317-4cc1-b0a3-a061c0c9327d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:38.203220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7990c71-6317-4cc1-b0a3-a061c0c9327d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:38.214390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7990c71-6317-4cc1-b0a3-a061c0c9327d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-d7990c71-6317-4cc1-b0a3-a061c0c9327d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:38.223023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:38.224078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:38.225075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:38.227446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 144/289] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:38.232869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b5ef6-c6c2-4b7d-9d59-13c27cbb87c5 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:38.284307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:38.388579 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b5ef6-c6c2-4b7d-9d59-13c27cbb87c5 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:38.436644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24fd05-8705-41c6-a9f5-aac9de0fa84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:38.437322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24fd05-8705-41c6-a9f5-aac9de0fa84e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:38.437615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24fd05-8705-41c6-a9f5-aac9de0fa84e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:38.437896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24fd05-8705-41c6-a9f5-aac9de0fa84e None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:38.438662 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24fd05-8705-41c6-a9f5-aac9de0fa84e None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:38.438962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24fd05-8705-41c6-a9f5-aac9de0fa84e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:38.441301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24fd05-8705-41c6-a9f5-aac9de0fa84e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:38.453614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 146/290] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:38.460404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7990c71-6317-4cc1-b0a3-a061c0c9327d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:38.584890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7990c71-6317-4cc1-b0a3-a061c0c9327d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:38.586664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:38.632598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a9b1a-0aec-4c1d-bb6d-f088666c0c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:38.633752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a9b1a-0aec-4c1d-bb6d-f088666c0c97 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:38.634184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a9b1a-0aec-4c1d-bb6d-f088666c0c97 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:38.634604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a9b1a-0aec-4c1d-bb6d-f088666c0c97 None admin] PATH_INFO: `/v3/users/2c11d79ccdb5455693485ab60192fcdb` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:38.638284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a9b1a-0aec-4c1d-bb6d-f088666c0c97 None admin] RBAC: Authorizing `identity:get_user(user_id=2c11d79ccdb5455693485ab60192fcdb)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:38.639951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a9b1a-0aec-4c1d-bb6d-f088666c0c97 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:38.642111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 145/291] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:38 2022] GET /identity/v3/users/2c11d79ccdb5455693485ab60192fcdb => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:38.647717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24fd05-8705-41c6-a9f5-aac9de0fa84e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:38.797987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:38.807857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24fd05-8705-41c6-a9f5-aac9de0fa84e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:38.858415 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c7355-c347-4d2f-9cc9-3c445a5e9590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:38.859106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c7355-c347-4d2f-9cc9-3c445a5e9590 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:38.859390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c7355-c347-4d2f-9cc9-3c445a5e9590 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:38.859671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c7355-c347-4d2f-9cc9-3c445a5e9590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:38.948996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c7355-c347-4d2f-9cc9-3c445a5e9590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:38.950447 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c7355-c347-4d2f-9cc9-3c445a5e9590 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:39.061962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:39.069332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 147/292] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:39.073996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a9b1a-0aec-4c1d-bb6d-f088666c0c97 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:39.191487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a9b1a-0aec-4c1d-bb6d-f088666c0c97 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:39.234372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd61b0-4e47-4b0d-b982-f164c62400d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:39.235158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd61b0-4e47-4b0d-b982-f164c62400d8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:39.235441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd61b0-4e47-4b0d-b982-f164c62400d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:39.235717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd61b0-4e47-4b0d-b982-f164c62400d8 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:39.237441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd61b0-4e47-4b0d-b982-f164c62400d8 None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:39.238935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd61b0-4e47-4b0d-b982-f164c62400d8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:39.241017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 146/293] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:39 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:39.246717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c7355-c347-4d2f-9cc9-3c445a5e9590 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:39.370078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c7355-c347-4d2f-9cc9-3c445a5e9590 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:39.409566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a00997-2a85-4543-bd89-aca4d7974e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:39.410266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a00997-2a85-4543-bd89-aca4d7974e41 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:39.410546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a00997-2a85-4543-bd89-aca4d7974e41 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:39.410825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a00997-2a85-4543-bd89-aca4d7974e41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:39.411943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a00997-2a85-4543-bd89-aca4d7974e41 None admin] RBAC: Adding query filter params (role.id=2e1db0cf27684f389b9320673721684d, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f, user.id=2c11d79ccdb5455693485ab60192fcdb) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:39.412257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a00997-2a85-4543-bd89-aca4d7974e41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:39.413718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a00997-2a85-4543-bd89-aca4d7974e41 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:39.424335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 148/294] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:39 2022] GET /identity/v3/role_assignments?user.id=2c11d79ccdb5455693485ab60192fcdb&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=2e1db0cf27684f389b9320673721684d => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:39.694321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc9129-0a63-465c-be63-ee98dce92be9 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:39.694847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc9129-0a63-465c-be63-ee98dce92be9 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:39.694847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc9129-0a63-465c-be63-ee98dce92be9 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:39.696450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 147/295] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:39.701464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75029cf9-53f8-43ce-b5b0-b05c5b257139 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:39.701785 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75029cf9-53f8-43ce-b5b0-b05c5b257139 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:39.702056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75029cf9-53f8-43ce-b5b0-b05c5b257139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:39.719633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-75029cf9-53f8-43ce-b5b0-b05c5b257139 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:39.721348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-75029cf9-53f8-43ce-b5b0-b05c5b257139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:39.749314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae912a-9f2c-400e-b849-615101539130 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:39.749684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae912a-9f2c-400e-b849-615101539130 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:39.749948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae912a-9f2c-400e-b849-615101539130 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:39.751663 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 148/296] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:39.761725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e94541-5430-47cb-9dbf-936a7b9167e8 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:39.762381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e94541-5430-47cb-9dbf-936a7b9167e8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:39.762759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e94541-5430-47cb-9dbf-936a7b9167e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:39.781286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:39.786805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-41e94541-5430-47cb-9dbf-936a7b9167e8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:39.788262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-41e94541-5430-47cb-9dbf-936a7b9167e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:39.804569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 149/297] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:39.829400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75029cf9-53f8-43ce-b5b0-b05c5b257139 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:39.838589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:39.897934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 149/298] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:39.921797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e94541-5430-47cb-9dbf-936a7b9167e8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:39.954454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75029cf9-53f8-43ce-b5b0-b05c5b257139 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:39.997874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433919c-73de-4c6e-a662-f96087a8c754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:39.998772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433919c-73de-4c6e-a662-f96087a8c754 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:39.999365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433919c-73de-4c6e-a662-f96087a8c754 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:39.999718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433919c-73de-4c6e-a662-f96087a8c754 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.002450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433919c-73de-4c6e-a662-f96087a8c754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:40.004004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433919c-73de-4c6e-a662-f96087a8c754 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:40.005808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 150/299] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:40.007548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f617188-579f-426c-9f38-b0fd5cbea7d1 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.007839 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f617188-579f-426c-9f38-b0fd5cbea7d1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.008115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f617188-579f-426c-9f38-b0fd5cbea7d1 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.010267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 151/300] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:40.012191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f617188-579f-426c-9f38-b0fd5cbea7d1 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:40.034826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e94541-5430-47cb-9dbf-936a7b9167e8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:40.069049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:40.073504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835a71a-0671-47d0-9b75-6201fe2d824d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:40.074305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835a71a-0671-47d0-9b75-6201fe2d824d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.074578 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835a71a-0671-47d0-9b75-6201fe2d824d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.074850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835a71a-0671-47d0-9b75-6201fe2d824d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.086280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835a71a-0671-47d0-9b75-6201fe2d824d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-2835a71a-0671-47d0-9b75-6201fe2d824d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:40.094060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:40.102381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 150/301] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:40.103377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f07ed-d770-4dd0-a899-be60bb0715fc None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.104131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f07ed-d770-4dd0-a899-be60bb0715fc None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.104131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f07ed-d770-4dd0-a899-be60bb0715fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.120066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a64f07ed-d770-4dd0-a899-be60bb0715fc None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:40.121259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a64f07ed-d770-4dd0-a899-be60bb0715fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:40.122358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f617188-579f-426c-9f38-b0fd5cbea7d1 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:40.147908 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:40.159123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49d472-46a9-469c-b999-c65cf0d2b0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:40.159868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49d472-46a9-469c-b999-c65cf0d2b0e8 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.160142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49d472-46a9-469c-b999-c65cf0d2b0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.160413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49d472-46a9-469c-b999-c65cf0d2b0e8 None admin] PATH_INFO: `/v3/groups` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.161420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49d472-46a9-469c-b999-c65cf0d2b0e8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:40.163046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49d472-46a9-469c-b999-c65cf0d2b0e8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:40.180046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 152/302] 10.222.0.66 () {60 vars in 1134 bytes} [Fri Jul 1 19:39:40 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:40.197619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49d472-46a9-469c-b999-c65cf0d2b0e8 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:40.211500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 151/303] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:40.237711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f07ed-d770-4dd0-a899-be60bb0715fc None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:40.330602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49d472-46a9-469c-b999-c65cf0d2b0e8 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:40.351140 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:40.383341 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a550b89-deb4-4357-8bd0-8af6b3e2f6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:40.384509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a550b89-deb4-4357-8bd0-8af6b3e2f6f4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.385279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a550b89-deb4-4357-8bd0-8af6b3e2f6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.385875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a550b89-deb4-4357-8bd0-8af6b3e2f6f4 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.386867 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a550b89-deb4-4357-8bd0-8af6b3e2f6f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:40.387448 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a550b89-deb4-4357-8bd0-8af6b3e2f6f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:40.389537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a550b89-deb4-4357-8bd0-8af6b3e2f6f4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:40.396638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f07ed-d770-4dd0-a899-be60bb0715fc None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:40.400947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 153/304] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:40.407703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a550b89-deb4-4357-8bd0-8af6b3e2f6f4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:40.434694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bdbc8-485d-474a-8e1b-2ab7fe0942d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:40.435551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bdbc8-485d-474a-8e1b-2ab7fe0942d3 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.435838 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bdbc8-485d-474a-8e1b-2ab7fe0942d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.436117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bdbc8-485d-474a-8e1b-2ab7fe0942d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.447647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bdbc8-485d-474a-8e1b-2ab7fe0942d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-456bdbc8-485d-474a-8e1b-2ab7fe0942d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:40.457785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:40.458849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:40.459869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 152/305] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:39:40.467377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bdbc8-485d-474a-8e1b-2ab7fe0942d3 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:40.496537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:40.540524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a550b89-deb4-4357-8bd0-8af6b3e2f6f4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:40.582012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f2949-604a-41d8-bc34-7ed4c065c915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:40.582917 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f2949-604a-41d8-bc34-7ed4c065c915 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.583194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f2949-604a-41d8-bc34-7ed4c065c915 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.583612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f2949-604a-41d8-bc34-7ed4c065c915 None admin] PATH_INFO: `/v3/users/8c9fc8762f4d4652861ba0a3ef1c7c60` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.598867 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f2949-604a-41d8-bc34-7ed4c065c915 None admin] RBAC: Authorizing `identity:get_user(user_id=8c9fc8762f4d4652861ba0a3ef1c7c60)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:40.601090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f2949-604a-41d8-bc34-7ed4c065c915 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:40.604064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 154/306] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:40 2022] GET /identity/v3/users/8c9fc8762f4d4652861ba0a3ef1c7c60 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:40.605343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cddc0-9120-4cc4-b441-0e1aae10613c None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.605701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cddc0-9120-4cc4-b441-0e1aae10613c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.605978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cddc0-9120-4cc4-b441-0e1aae10613c None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.606785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bdbc8-485d-474a-8e1b-2ab7fe0942d3 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:40.607219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 155/307] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:40.610461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cddc0-9120-4cc4-b441-0e1aae10613c None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:40.644612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:40.647815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0d42e-e4d5-4df9-8a0f-5e8da9a7f993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:40.648657 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0d42e-e4d5-4df9-8a0f-5e8da9a7f993 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.649093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0d42e-e4d5-4df9-8a0f-5e8da9a7f993 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.649367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0d42e-e4d5-4df9-8a0f-5e8da9a7f993 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.650150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0d42e-e4d5-4df9-8a0f-5e8da9a7f993 None admin] RBAC: Adding query filter params (name=member) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:40.650459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0d42e-e4d5-4df9-8a0f-5e8da9a7f993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:40.651864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0d42e-e4d5-4df9-8a0f-5e8da9a7f993 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:40.660704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 153/308] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:40.662215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee2d27-8cca-4803-848d-0e46fb7d5407 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.662506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee2d27-8cca-4803-848d-0e46fb7d5407 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.662781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee2d27-8cca-4803-848d-0e46fb7d5407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.678888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-bbee2d27-8cca-4803-848d-0e46fb7d5407 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:40.680144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-bbee2d27-8cca-4803-848d-0e46fb7d5407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:40.729394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cddc0-9120-4cc4-b441-0e1aae10613c None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:40.753641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 154/309] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:40.753641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee2d27-8cca-4803-848d-0e46fb7d5407 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:40.769167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:40.773891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b2268-1bdf-4917-998b-41d9de126ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:40.774522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b2268-1bdf-4917-998b-41d9de126ac4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.774805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b2268-1bdf-4917-998b-41d9de126ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.775082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b2268-1bdf-4917-998b-41d9de126ac4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.776279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b2268-1bdf-4917-998b-41d9de126ac4 None admin] RBAC: Adding query filter params (role.id=0c5069e08baa4820b7154564a801ba7e, user.id=8c9fc8762f4d4652861ba0a3ef1c7c60, scope.system=all) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:40.776591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b2268-1bdf-4917-998b-41d9de126ac4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:40.778063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b2268-1bdf-4917-998b-41d9de126ac4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:40.783935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 156/310] 10.222.0.66 () {56 vars in 1308 bytes} [Fri Jul 1 19:39:40 2022] GET /identity/v3/role_assignments?user.id=8c9fc8762f4d4652861ba0a3ef1c7c60&scope.system=all&role.id=0c5069e08baa4820b7154564a801ba7e => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:40.784576 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b2268-1bdf-4917-998b-41d9de126ac4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:40.886460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee2d27-8cca-4803-848d-0e46fb7d5407 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:40.913524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:40.920691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b2268-1bdf-4917-998b-41d9de126ac4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:40.931706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050e4f4-cc92-467a-b95d-e2773caf6b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:40.932966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050e4f4-cc92-467a-b95d-e2773caf6b0c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.933447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050e4f4-cc92-467a-b95d-e2773caf6b0c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.933940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050e4f4-cc92-467a-b95d-e2773caf6b0c None admin] PATH_INFO: `/v3/users/33104eb5a28b421984dbea9fa0bb91d7` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.935416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050e4f4-cc92-467a-b95d-e2773caf6b0c None admin] RBAC: Authorizing `identity:get_user(user_id=33104eb5a28b421984dbea9fa0bb91d7)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:40.937199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050e4f4-cc92-467a-b95d-e2773caf6b0c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:40.939241 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 155/311] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:40 2022] GET /identity/v3/users/33104eb5a28b421984dbea9fa0bb91d7 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:40.946491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050e4f4-cc92-467a-b95d-e2773caf6b0c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:40.960499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ec978-af09-453f-bb9c-a2f259a3038a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:40.961308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ec978-af09-453f-bb9c-a2f259a3038a None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:40.962105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ec978-af09-453f-bb9c-a2f259a3038a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:40.962105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ec978-af09-453f-bb9c-a2f259a3038a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:40.973556 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ec978-af09-453f-bb9c-a2f259a3038a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-167ec978-af09-453f-bb9c-a2f259a3038a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:40.981457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:40.982408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:40.983191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 157/312] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:40.991125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ec978-af09-453f-bb9c-a2f259a3038a None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:41.049479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:41.077961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050e4f4-cc92-467a-b95d-e2773caf6b0c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:41.121369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7376e-4787-408f-acc3-e97df0fc988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:41.122284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7376e-4787-408f-acc3-e97df0fc988b None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:41.122565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7376e-4787-408f-acc3-e97df0fc988b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:41.122842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7376e-4787-408f-acc3-e97df0fc988b None admin] PATH_INFO: `/v3/system/users/33104eb5a28b421984dbea9fa0bb91d7/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:41.124822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7376e-4787-408f-acc3-e97df0fc988b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=33104eb5a28b421984dbea9fa0bb91d7, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:41.126526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7376e-4787-408f-acc3-e97df0fc988b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:41.128054 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ec978-af09-453f-bb9c-a2f259a3038a None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:41.137936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 156/313] 10.222.0.66 () {58 vars in 1334 bytes} [Fri Jul 1 19:39:40 2022] PUT /identity/v3/system/users/33104eb5a28b421984dbea9fa0bb91d7/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:39:41.170620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d98ef-b369-43b1-8448-142ab8f05eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:41.171317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d98ef-b369-43b1-8448-142ab8f05eba None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:41.171600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d98ef-b369-43b1-8448-142ab8f05eba None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:41.171876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d98ef-b369-43b1-8448-142ab8f05eba None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:41.172641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d98ef-b369-43b1-8448-142ab8f05eba None admin] RBAC: Adding query filter params (name=reader) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:41.173010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d98ef-b369-43b1-8448-142ab8f05eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:41.175073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d98ef-b369-43b1-8448-142ab8f05eba None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:41.184958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 158/314] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:41.191912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7376e-4787-408f-acc3-e97df0fc988b None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:41.307083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7376e-4787-408f-acc3-e97df0fc988b None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:41.344103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0604aa6-4824-4db5-a36b-0a6160374cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:41.344984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0604aa6-4824-4db5-a36b-0a6160374cf6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:41.345263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0604aa6-4824-4db5-a36b-0a6160374cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:41.345568 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0604aa6-4824-4db5-a36b-0a6160374cf6 None admin] PATH_INFO: `/v3/users/143d447710be462788b05d34fabb23f9` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:41.346943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0604aa6-4824-4db5-a36b-0a6160374cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=143d447710be462788b05d34fabb23f9)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:41.348400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0604aa6-4824-4db5-a36b-0a6160374cf6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:41.350161 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 157/315] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:41 2022] GET /identity/v3/users/143d447710be462788b05d34fabb23f9 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:41.355744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d98ef-b369-43b1-8448-142ab8f05eba None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:41.377837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:41.467103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d98ef-b369-43b1-8448-142ab8f05eba None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:41.503793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02c90c-9d5a-43df-b0a5-8145169f2513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:41.503793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02c90c-9d5a-43df-b0a5-8145169f2513 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:41.503793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02c90c-9d5a-43df-b0a5-8145169f2513 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:41.503793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02c90c-9d5a-43df-b0a5-8145169f2513 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:41.575373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02c90c-9d5a-43df-b0a5-8145169f2513 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:41.576856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02c90c-9d5a-43df-b0a5-8145169f2513 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:41.602007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:41.683380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 159/316] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:41.685195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0604aa6-4824-4db5-a36b-0a6160374cf6 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:41.698817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:41.793203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0604aa6-4824-4db5-a36b-0a6160374cf6 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:41.828740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b5b6f-12e1-4d22-81e1-01c18bfb4831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:41.830136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b5b6f-12e1-4d22-81e1-01c18bfb4831 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:41.830813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b5b6f-12e1-4d22-81e1-01c18bfb4831 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:41.831290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b5b6f-12e1-4d22-81e1-01c18bfb4831 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:41.832755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b5b6f-12e1-4d22-81e1-01c18bfb4831 None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:41.834451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b5b6f-12e1-4d22-81e1-01c18bfb4831 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:41.836694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 158/317] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:41 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:41.842360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02c90c-9d5a-43df-b0a5-8145169f2513 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:41.947711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02c90c-9d5a-43df-b0a5-8145169f2513 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:41.966508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:41.986723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69e381-e352-404a-aa70-a1045c42a5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:41.987384 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69e381-e352-404a-aa70-a1045c42a5e1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:41.987666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69e381-e352-404a-aa70-a1045c42a5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:41.987942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69e381-e352-404a-aa70-a1045c42a5e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:41.988780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69e381-e352-404a-aa70-a1045c42a5e1 None admin] RBAC: Adding query filter params (role.id=0c5069e08baa4820b7154564a801ba7e, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f, user.id=143d447710be462788b05d34fabb23f9) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:41.989095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69e381-e352-404a-aa70-a1045c42a5e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:41.990561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69e381-e352-404a-aa70-a1045c42a5e1 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:42.001713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 160/318] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:39:41 2022] GET /identity/v3/role_assignments?user.id=143d447710be462788b05d34fabb23f9&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=0c5069e08baa4820b7154564a801ba7e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:42.352855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06814eb-342e-4b85-a41f-978978739457 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:42.353255 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06814eb-342e-4b85-a41f-978978739457 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:42.353433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06814eb-342e-4b85-a41f-978978739457 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:42.354971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 159/319] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:42.359967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1f105-8cd6-44a5-9ee7-349bfc6e08b4 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:42.360420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1f105-8cd6-44a5-9ee7-349bfc6e08b4 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:42.360581 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1f105-8cd6-44a5-9ee7-349bfc6e08b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:42.377221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6ab1f105-8cd6-44a5-9ee7-349bfc6e08b4 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:42.378426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6ab1f105-8cd6-44a5-9ee7-349bfc6e08b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:42.447683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 161/320] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:42.472095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06814eb-342e-4b85-a41f-978978739457 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:42.559209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:42.583799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06814eb-342e-4b85-a41f-978978739457 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:42.625776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132ce4d-1080-4703-ab4b-65db1aad95eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:42.626492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132ce4d-1080-4703-ab4b-65db1aad95eb None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:42.626789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132ce4d-1080-4703-ab4b-65db1aad95eb None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:42.627064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132ce4d-1080-4703-ab4b-65db1aad95eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:42.628228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132ce4d-1080-4703-ab4b-65db1aad95eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:42.629753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132ce4d-1080-4703-ab4b-65db1aad95eb None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:42.631545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 160/321] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:39:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:42.637119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1f105-8cd6-44a5-9ee7-349bfc6e08b4 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:42.739089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1f105-8cd6-44a5-9ee7-349bfc6e08b4 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:42.753843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:42.776966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849aef7c-6913-488b-a39f-901ea4c10faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:42.777887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aef7c-6913-488b-a39f-901ea4c10faa None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:42.778336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aef7c-6913-488b-a39f-901ea4c10faa None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:42.778751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aef7c-6913-488b-a39f-901ea4c10faa None admin] PATH_INFO: `/v3/groups` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:42.779587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aef7c-6913-488b-a39f-901ea4c10faa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:42.781167 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aef7c-6913-488b-a39f-901ea4c10faa None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:42.797131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 162/322] 10.222.0.66 () {60 vars in 1134 bytes} [Fri Jul 1 19:39:42 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:39:42.934648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc3512-df9a-4a8a-a56e-f18fc26986b9 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:42.935135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc3512-df9a-4a8a-a56e-f18fc26986b9 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:42.935135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc3512-df9a-4a8a-a56e-f18fc26986b9 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:42.936631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 161/323] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:42.942656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359318d-1539-49c5-9eb3-a8bc94964454 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:42.942946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359318d-1539-49c5-9eb3-a8bc94964454 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:42.943211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359318d-1539-49c5-9eb3-a8bc94964454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:42.961055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9359318d-1539-49c5-9eb3-a8bc94964454 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:42.962309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9359318d-1539-49c5-9eb3-a8bc94964454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:43.034886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 163/324] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:43.058295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc3512-df9a-4a8a-a56e-f18fc26986b9 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:43.178091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc3512-df9a-4a8a-a56e-f18fc26986b9 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:43.216943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1693201-f54e-40bc-8810-0a177af67a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:43.218094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1693201-f54e-40bc-8810-0a177af67a7d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:43.218627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1693201-f54e-40bc-8810-0a177af67a7d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:43.219119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1693201-f54e-40bc-8810-0a177af67a7d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:43.231342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1693201-f54e-40bc-8810-0a177af67a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-a1693201-f54e-40bc-8810-0a177af67a7d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:43.240329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:43.241210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:43.242003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:43.244620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 162/325] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:43.250240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359318d-1539-49c5-9eb3-a8bc94964454 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:43.305597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621ed26-e787-433c-838c-92805322345c None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:43.305911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621ed26-e787-433c-838c-92805322345c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:43.306160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621ed26-e787-433c-838c-92805322345c None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:43.308291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 163/326] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:39:43.312647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634161f-1a91-4162-bf22-19d5ed9f6ad2 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:43.312933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634161f-1a91-4162-bf22-19d5ed9f6ad2 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:43.313213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634161f-1a91-4162-bf22-19d5ed9f6ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:43.327811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:43.335612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9634161f-1a91-4162-bf22-19d5ed9f6ad2 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:43.337200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9634161f-1a91-4162-bf22-19d5ed9f6ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:43.379837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359318d-1539-49c5-9eb3-a8bc94964454 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:43.381881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:43.429848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a63716-1925-4b9d-860c-4e5fc6f0e1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:43.430796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a63716-1925-4b9d-860c-4e5fc6f0e1f4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:43.431224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a63716-1925-4b9d-860c-4e5fc6f0e1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:43.431658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a63716-1925-4b9d-860c-4e5fc6f0e1f4 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:43.431892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 164/327] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:43.432888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a63716-1925-4b9d-860c-4e5fc6f0e1f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:43.433336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a63716-1925-4b9d-860c-4e5fc6f0e1f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:43.434923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a63716-1925-4b9d-860c-4e5fc6f0e1f4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:43.446774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 164/328] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:43.452131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634161f-1a91-4162-bf22-19d5ed9f6ad2 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:43.460628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a63716-1925-4b9d-860c-4e5fc6f0e1f4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:43.581374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a63716-1925-4b9d-860c-4e5fc6f0e1f4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:43.583397 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634161f-1a91-4162-bf22-19d5ed9f6ad2 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:43.620143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebd0f7-3ea5-4128-b60c-383ce5a30a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:43.621111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebd0f7-3ea5-4128-b60c-383ce5a30a97 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:43.621494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebd0f7-3ea5-4128-b60c-383ce5a30a97 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:43.621890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebd0f7-3ea5-4128-b60c-383ce5a30a97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:43.624067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc50b9c-397b-4fc4-bdad-1eee803ce7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:43.625208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc50b9c-397b-4fc4-bdad-1eee803ce7bf None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:43.625772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc50b9c-397b-4fc4-bdad-1eee803ce7bf None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:43.626224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc50b9c-397b-4fc4-bdad-1eee803ce7bf None admin] PATH_INFO: `/v3/users/8c9fc8762f4d4652861ba0a3ef1c7c60` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:43.627893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc50b9c-397b-4fc4-bdad-1eee803ce7bf None admin] RBAC: Authorizing `identity:get_user(user_id=8c9fc8762f4d4652861ba0a3ef1c7c60)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:43.629829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc50b9c-397b-4fc4-bdad-1eee803ce7bf None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:43.631939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 165/329] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:43 2022] GET /identity/v3/users/8c9fc8762f4d4652861ba0a3ef1c7c60 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:43.634269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebd0f7-3ea5-4128-b60c-383ce5a30a97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:43.637447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc50b9c-397b-4fc4-bdad-1eee803ce7bf None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:43.641430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-78ebd0f7-3ea5-4128-b60c-383ce5a30a97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:43.642892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:43.643876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:43.645691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 165/330] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:39:43.649722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebd0f7-3ea5-4128-b60c-383ce5a30a97 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:43.683465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:43.806609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc50b9c-397b-4fc4-bdad-1eee803ce7bf None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:43.819340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebd0f7-3ea5-4128-b60c-383ce5a30a97 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:43.851327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d95e88-4cb3-4965-b1b0-c46c4fb39b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:43.852540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d95e88-4cb3-4965-b1b0-c46c4fb39b90 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:43.853000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d95e88-4cb3-4965-b1b0-c46c4fb39b90 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:43.853451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d95e88-4cb3-4965-b1b0-c46c4fb39b90 None admin] PATH_INFO: `/v3/system/users/8c9fc8762f4d4652861ba0a3ef1c7c60/roles/0c5069e08baa4820b7154564a801ba7e` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:43.855492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d95e88-4cb3-4965-b1b0-c46c4fb39b90 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8c9fc8762f4d4652861ba0a3ef1c7c60, role_id=0c5069e08baa4820b7154564a801ba7e)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:43.857190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d95e88-4cb3-4965-b1b0-c46c4fb39b90 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:43.868750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354253ae-1c04-42f2-bb75-d4ed332c5364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:43.869605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354253ae-1c04-42f2-bb75-d4ed332c5364 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:43.869973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354253ae-1c04-42f2-bb75-d4ed332c5364 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:43.870320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354253ae-1c04-42f2-bb75-d4ed332c5364 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:43.871150 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354253ae-1c04-42f2-bb75-d4ed332c5364 None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:43.871583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354253ae-1c04-42f2-bb75-d4ed332c5364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:43.873020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354253ae-1c04-42f2-bb75-d4ed332c5364 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:43.875473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 166/331] 10.222.0.66 () {58 vars in 1334 bytes} [Fri Jul 1 19:39:43 2022] PUT /identity/v3/system/users/8c9fc8762f4d4652861ba0a3ef1c7c60/roles/0c5069e08baa4820b7154564a801ba7e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:39:43.884559 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 166/332] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:43.890731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d95e88-4cb3-4965-b1b0-c46c4fb39b90 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:43.966416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:44.004968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d95e88-4cb3-4965-b1b0-c46c4fb39b90 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:44.043172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33837d51-d004-452f-8b1a-0c9539136aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:44.044063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33837d51-d004-452f-8b1a-0c9539136aa0 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:44.044342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33837d51-d004-452f-8b1a-0c9539136aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:44.044614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33837d51-d004-452f-8b1a-0c9539136aa0 None admin] PATH_INFO: `/v3/users/33104eb5a28b421984dbea9fa0bb91d7` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:44.045956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33837d51-d004-452f-8b1a-0c9539136aa0 None admin] RBAC: Authorizing `identity:get_user(user_id=33104eb5a28b421984dbea9fa0bb91d7)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:44.047434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33837d51-d004-452f-8b1a-0c9539136aa0 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:44.050097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 167/333] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:43 2022] GET /identity/v3/users/33104eb5a28b421984dbea9fa0bb91d7 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:44.055979 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354253ae-1c04-42f2-bb75-d4ed332c5364 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:44.154109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:44.166175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354253ae-1c04-42f2-bb75-d4ed332c5364 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:44.201608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6330e4-8126-489c-b270-aaa28754a3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:44.202287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6330e4-8126-489c-b270-aaa28754a3ec None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:44.202565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6330e4-8126-489c-b270-aaa28754a3ec None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:44.202839 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6330e4-8126-489c-b270-aaa28754a3ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:44.203684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6330e4-8126-489c-b270-aaa28754a3ec None admin] RBAC: Adding query filter params (role.id=2e1db0cf27684f389b9320673721684d, user.id=33104eb5a28b421984dbea9fa0bb91d7, scope.system=all) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:44.203989 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6330e4-8126-489c-b270-aaa28754a3ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:44.205532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6330e4-8126-489c-b270-aaa28754a3ec None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:44.218334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 167/334] 10.222.0.66 () {56 vars in 1308 bytes} [Fri Jul 1 19:39:44 2022] GET /identity/v3/role_assignments?user.id=33104eb5a28b421984dbea9fa0bb91d7&scope.system=all&role.id=2e1db0cf27684f389b9320673721684d => generated 510 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:45.497872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852ce10-0dfd-41da-a74a-2d8a2b55690f None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:45.498377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852ce10-0dfd-41da-a74a-2d8a2b55690f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:45.498643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852ce10-0dfd-41da-a74a-2d8a2b55690f None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:45.500128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 168/335] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:45.510111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94e654-48c0-412f-861b-63e90f4e41a3 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:45.510565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94e654-48c0-412f-861b-63e90f4e41a3 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:45.510754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94e654-48c0-412f-861b-63e90f4e41a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:45.528425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-8c94e654-48c0-412f-861b-63e90f4e41a3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:45.533722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8c94e654-48c0-412f-861b-63e90f4e41a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:45.606062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 168/336] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:45.650621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852ce10-0dfd-41da-a74a-2d8a2b55690f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:45.791883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852ce10-0dfd-41da-a74a-2d8a2b55690f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:45.832724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4f1a7-89e2-4450-b472-a264486ce94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:45.833525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4f1a7-89e2-4450-b472-a264486ce94b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:45.833823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4f1a7-89e2-4450-b472-a264486ce94b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:45.834089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4f1a7-89e2-4450-b472-a264486ce94b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:45.836848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:45.850022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4f1a7-89e2-4450-b472-a264486ce94b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-2ab4f1a7-89e2-4450-b472-a264486ce94b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:45.858588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:45.860693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:45.862761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:45.864993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 169/337] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:45.874359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94e654-48c0-412f-861b-63e90f4e41a3 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:45.937538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cb44f-59d4-4ba0-8a93-5807e342d647 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:45.937900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cb44f-59d4-4ba0-8a93-5807e342d647 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:45.938095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cb44f-59d4-4ba0-8a93-5807e342d647 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:45.939799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 170/338] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:45.959417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665cb5-ebaf-4e08-9044-ded40cbfa6c1 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:45.959704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665cb5-ebaf-4e08-9044-ded40cbfa6c1 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:45.959952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da665cb5-ebaf-4e08-9044-ded40cbfa6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:45.984292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:46.007826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-da665cb5-ebaf-4e08-9044-ded40cbfa6c1 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:46.011387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-da665cb5-ebaf-4e08-9044-ded40cbfa6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:46.025361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94e654-48c0-412f-861b-63e90f4e41a3 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:46.084293 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4c0c-6f4c-466e-8fbd-f4e0dcfa3ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:46.085007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4c0c-6f4c-466e-8fbd-f4e0dcfa3ec9 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:46.085262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4c0c-6f4c-466e-8fbd-f4e0dcfa3ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:46.085534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4c0c-6f4c-466e-8fbd-f4e0dcfa3ec9 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:46.086340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a4c0c-6f4c-466e-8fbd-f4e0dcfa3ec9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:46.086640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a4c0c-6f4c-466e-8fbd-f4e0dcfa3ec9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:46.088023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a4c0c-6f4c-466e-8fbd-f4e0dcfa3ec9 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:46.107977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 169/339] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:46.113670 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4c0c-6f4c-466e-8fbd-f4e0dcfa3ec9 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:46.186954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 171/340] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:46.214704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da665cb5-ebaf-4e08-9044-ded40cbfa6c1 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:46.263330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4c0c-6f4c-466e-8fbd-f4e0dcfa3ec9 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:46.316237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe069b-746f-4d4f-9d0b-1b57feaf05cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:46.317429 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe069b-746f-4d4f-9d0b-1b57feaf05cc None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:46.319081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe069b-746f-4d4f-9d0b-1b57feaf05cc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:46.320348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe069b-746f-4d4f-9d0b-1b57feaf05cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:46.329116 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:46.457890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe069b-746f-4d4f-9d0b-1b57feaf05cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:46.467022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe069b-746f-4d4f-9d0b-1b57feaf05cc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:46.473552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da665cb5-ebaf-4e08-9044-ded40cbfa6c1 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:46.526275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:46.543985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a2dbd-1826-4370-ab7a-30a158781f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:46.545149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a2dbd-1826-4370-ab7a-30a158781f62 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:46.545672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a2dbd-1826-4370-ab7a-30a158781f62 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:46.546262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a2dbd-1826-4370-ab7a-30a158781f62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:46.577020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a2dbd-1826-4370-ab7a-30a158781f62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-207a2dbd-1826-4370-ab7a-30a158781f62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:46.585373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:46.590426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:46.591192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:46.599027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 172/341] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:46.606550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a2dbd-1826-4370-ab7a-30a158781f62 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:46.703177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 170/342] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 19:39:46.705239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0cd1e-18fb-4d54-9e6d-702ce768ba1a None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:46.705350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0cd1e-18fb-4d54-9e6d-702ce768ba1a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:46.705639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0cd1e-18fb-4d54-9e6d-702ce768ba1a None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:46.714328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 171/343] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:46.715388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0cd1e-18fb-4d54-9e6d-702ce768ba1a None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:46.746275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a2dbd-1826-4370-ab7a-30a158781f62 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:46.782813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450099a-f732-41ec-a96d-24865e9fb34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:46.784278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:46.785103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450099a-f732-41ec-a96d-24865e9fb34f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:46.785527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450099a-f732-41ec-a96d-24865e9fb34f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:46.785993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450099a-f732-41ec-a96d-24865e9fb34f None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:46.787137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450099a-f732-41ec-a96d-24865e9fb34f None admin] RBAC: Adding query filter params (name=member) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:46.787580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450099a-f732-41ec-a96d-24865e9fb34f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:46.789136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450099a-f732-41ec-a96d-24865e9fb34f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:46.799517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 173/344] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:46.801026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce1526-23b3-4d98-9d67-14a73005ca7c None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:46.801443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce1526-23b3-4d98-9d67-14a73005ca7c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:46.801887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce1526-23b3-4d98-9d67-14a73005ca7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:46.830983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-07ce1526-23b3-4d98-9d67-14a73005ca7c None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:46.833138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-07ce1526-23b3-4d98-9d67-14a73005ca7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:46.917166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0cd1e-18fb-4d54-9e6d-702ce768ba1a None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:46.921958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:46.969891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 174/345] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:46.971192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce1526-23b3-4d98-9d67-14a73005ca7c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:46.990577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1809c-4d1f-42e6-9e5f-f31587cb4ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:46.991458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1809c-4d1f-42e6-9e5f-f31587cb4ed8 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:46.991733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1809c-4d1f-42e6-9e5f-f31587cb4ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:46.991998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1809c-4d1f-42e6-9e5f-f31587cb4ed8 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:46.994659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1809c-4d1f-42e6-9e5f-f31587cb4ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:46.996117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1809c-4d1f-42e6-9e5f-f31587cb4ed8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:46.998010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 172/346] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:46 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:47.004725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1809c-4d1f-42e6-9e5f-f31587cb4ed8 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:47.105007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce1526-23b3-4d98-9d67-14a73005ca7c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:47.124986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1809c-4d1f-42e6-9e5f-f31587cb4ed8 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:47.137955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:47.152961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fde85-151a-427a-9fa8-1360811d9316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:47.152961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fde85-151a-427a-9fa8-1360811d9316 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.152961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fde85-151a-427a-9fa8-1360811d9316 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.152961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fde85-151a-427a-9fa8-1360811d9316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.169053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb922f-8392-4a4e-9e00-807d04a59954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:47.170491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb922f-8392-4a4e-9e00-807d04a59954 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.171198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb922f-8392-4a4e-9e00-807d04a59954 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.171615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb922f-8392-4a4e-9e00-807d04a59954 None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.180187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb922f-8392-4a4e-9e00-807d04a59954 None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:47.182053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb922f-8392-4a4e-9e00-807d04a59954 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:47.183885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 173/347] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:47 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:47.185326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44620f-ea70-4aaa-9b86-c8e672f2890b None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.185627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44620f-ea70-4aaa-9b86-c8e672f2890b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.185904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44620f-ea70-4aaa-9b86-c8e672f2890b None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.187104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 174/348] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:47.188016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44620f-ea70-4aaa-9b86-c8e672f2890b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:47.217064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:47.245401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fde85-151a-427a-9fa8-1360811d9316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:47.246856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fde85-151a-427a-9fa8-1360811d9316 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:47.323931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44620f-ea70-4aaa-9b86-c8e672f2890b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:47.375365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 175/349] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:47.377064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb47d7-9152-4786-a98a-fab9ff289728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:47.378275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb47d7-9152-4786-a98a-fab9ff289728 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.379800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49fa1c-391f-4491-8a12-bd3e9b50f93b None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.380407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb47d7-9152-4786-a98a-fab9ff289728 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.380820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49fa1c-391f-4491-8a12-bd3e9b50f93b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.381061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb47d7-9152-4786-a98a-fab9ff289728 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.381161 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49fa1c-391f-4491-8a12-bd3e9b50f93b None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.386804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 176/350] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:47.386804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49fa1c-391f-4491-8a12-bd3e9b50f93b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:47.399080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb47d7-9152-4786-a98a-fab9ff289728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-52bb47d7-9152-4786-a98a-fab9ff289728 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:47.407117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:47.408361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:47.409157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:47.412150 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 175/351] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:47.413774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2286c61-0502-4ecd-9618-4887867286fb None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.414252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2286c61-0502-4ecd-9618-4887867286fb None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.414721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2286c61-0502-4ecd-9618-4887867286fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.434204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c2286c61-0502-4ecd-9618-4887867286fb None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:47.435774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c2286c61-0502-4ecd-9618-4887867286fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:47.468161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:47.490579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:47.516517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49fa1c-391f-4491-8a12-bd3e9b50f93b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:47.535370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 176/352] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:47.536784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2286c61-0502-4ecd-9618-4887867286fb None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:47.569820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cebd6-1326-4b80-a462-bee5215986e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:47.571057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cebd6-1326-4b80-a462-bee5215986e9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.571483 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cebd6-1326-4b80-a462-bee5215986e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.571897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cebd6-1326-4b80-a462-bee5215986e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.572942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cebd6-1326-4b80-a462-bee5215986e9 None admin] RBAC: Adding query filter params (group.id=8e1642a69c37473e8e4d86777f84633a, role.id=b127031d393048d7ae9a35015113e82a, scope.project.id=574afa1590854c65a807df4589e2c460) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:47.573417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cebd6-1326-4b80-a462-bee5215986e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:47.574904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cebd6-1326-4b80-a462-bee5215986e9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:47.581626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 177/353] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 1 19:39:47 2022] GET /identity/v3/role_assignments?group.id=8e1642a69c37473e8e4d86777f84633a&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=b127031d393048d7ae9a35015113e82a => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:47.587226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8de356-6a2b-4787-93aa-ec72d863f967 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.587828 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8de356-6a2b-4787-93aa-ec72d863f967 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.588358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8de356-6a2b-4787-93aa-ec72d863f967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.608517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7c8de356-6a2b-4787-93aa-ec72d863f967 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:47.612099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7c8de356-6a2b-4787-93aa-ec72d863f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:47.659255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2286c61-0502-4ecd-9618-4887867286fb None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:47.698831 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 178/354] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:47.700047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8de356-6a2b-4787-93aa-ec72d863f967 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:47.706955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53467b6-02f7-4a98-82a6-b32bdfd763ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:47.707852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53467b6-02f7-4a98-82a6-b32bdfd763ca None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.708133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53467b6-02f7-4a98-82a6-b32bdfd763ca None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.708411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53467b6-02f7-4a98-82a6-b32bdfd763ca None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.709682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53467b6-02f7-4a98-82a6-b32bdfd763ca None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:47.711188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53467b6-02f7-4a98-82a6-b32bdfd763ca None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:47.712972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 177/355] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:47 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:47.714845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53467b6-02f7-4a98-82a6-b32bdfd763ca None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:47.760211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:47.810300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:47.844717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8de356-6a2b-4787-93aa-ec72d863f967 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:47.854233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53467b6-02f7-4a98-82a6-b32bdfd763ca None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:47.881538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ed0e6-b07b-4506-8c33-37a32ac60e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:47.882239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ed0e6-b07b-4506-8c33-37a32ac60e6e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.882525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ed0e6-b07b-4506-8c33-37a32ac60e6e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.882804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ed0e6-b07b-4506-8c33-37a32ac60e6e None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.883560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ed0e6-b07b-4506-8c33-37a32ac60e6e None admin] RBAC: Adding query filter params (name=member) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:47.883866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ed0e6-b07b-4506-8c33-37a32ac60e6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:47.885245 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ed0e6-b07b-4506-8c33-37a32ac60e6e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:47.892399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35016b5-6deb-48b9-9541-64637fd5e2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:47.893213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35016b5-6deb-48b9-9541-64637fd5e2bb None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:47.893493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35016b5-6deb-48b9-9541-64637fd5e2bb None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:47.893798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35016b5-6deb-48b9-9541-64637fd5e2bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:47.895306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 179/356] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:47.896054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ed0e6-b07b-4506-8c33-37a32ac60e6e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:47.905323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35016b5-6deb-48b9-9541-64637fd5e2bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-b35016b5-6deb-48b9-9541-64637fd5e2bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:47.913900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:47.914791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 178/357] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:47.916162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35016b5-6deb-48b9-9541-64637fd5e2bb None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:48.009001 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ed0e6-b07b-4506-8c33-37a32ac60e6e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:48.019504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:48.042260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35016b5-6deb-48b9-9541-64637fd5e2bb None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:48.053352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1e318-5c50-4098-b6bb-4f8878a56ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:48.054261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1e318-5c50-4098-b6bb-4f8878a56ce6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.054907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1e318-5c50-4098-b6bb-4f8878a56ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.055185 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1e318-5c50-4098-b6bb-4f8878a56ce6 None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:48.057285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1e318-5c50-4098-b6bb-4f8878a56ce6 None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:48.058780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1e318-5c50-4098-b6bb-4f8878a56ce6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:48.061539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 180/358] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:47 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:48.061539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1e318-5c50-4098-b6bb-4f8878a56ce6 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:48.066194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:48.085834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f23a45-faab-413c-be9d-5d8b954f54e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:48.086714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f23a45-faab-413c-be9d-5d8b954f54e1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.087007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f23a45-faab-413c-be9d-5d8b954f54e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.087279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f23a45-faab-413c-be9d-5d8b954f54e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:48.101496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f23a45-faab-413c-be9d-5d8b954f54e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-20f23a45-faab-413c-be9d-5d8b954f54e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:48.110305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:48.111789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 179/359] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:48.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f23a45-faab-413c-be9d-5d8b954f54e1 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:48.192748 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1e318-5c50-4098-b6bb-4f8878a56ce6 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:48.234272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482ba02-17f5-432c-a2d3-bb88e50e2ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:48.236013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482ba02-17f5-432c-a2d3-bb88e50e2ebd None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.236295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482ba02-17f5-432c-a2d3-bb88e50e2ebd None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.236660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482ba02-17f5-432c-a2d3-bb88e50e2ebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:48.247951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f23a45-faab-413c-be9d-5d8b954f54e1 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:48.290207 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c5357-fe0e-4634-817b-56c5ead139fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:48.290932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c5357-fe0e-4634-817b-56c5ead139fe None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.291215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c5357-fe0e-4634-817b-56c5ead139fe None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.291486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c5357-fe0e-4634-817b-56c5ead139fe None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:48.292241 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c5357-fe0e-4634-817b-56c5ead139fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:48.292544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c5357-fe0e-4634-817b-56c5ead139fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:48.294024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c5357-fe0e-4634-817b-56c5ead139fe None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:48.306435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 180/360] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:48.307570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe9f65-6a99-4b6b-9e59-5692bb710cd6 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.307857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe9f65-6a99-4b6b-9e59-5692bb710cd6 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.308136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe9f65-6a99-4b6b-9e59-5692bb710cd6 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:48.309380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 181/361] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:48.310103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe9f65-6a99-4b6b-9e59-5692bb710cd6 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:48.324348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482ba02-17f5-432c-a2d3-bb88e50e2ebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:48.325779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482ba02-17f5-432c-a2d3-bb88e50e2ebd None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:48.326467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:48.327903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:48.447326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe9f65-6a99-4b6b-9e59-5692bb710cd6 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:48.459767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 181/362] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:48.460622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482ba02-17f5-432c-a2d3-bb88e50e2ebd None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:48.497039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074815b8-f37e-48a0-9e5c-d244a3d2da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:48.497730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074815b8-f37e-48a0-9e5c-d244a3d2da5d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.498015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074815b8-f37e-48a0-9e5c-d244a3d2da5d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.498291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074815b8-f37e-48a0-9e5c-d244a3d2da5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:48.499126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074815b8-f37e-48a0-9e5c-d244a3d2da5d None admin] RBAC: Adding query filter params (group.id=8e1642a69c37473e8e4d86777f84633a, role.id=2e1db0cf27684f389b9320673721684d, scope.project.id=574afa1590854c65a807df4589e2c460) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:48.499561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074815b8-f37e-48a0-9e5c-d244a3d2da5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:48.502232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074815b8-f37e-48a0-9e5c-d244a3d2da5d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:48.511870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 182/363] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 1 19:39:48 2022] GET /identity/v3/role_assignments?group.id=8e1642a69c37473e8e4d86777f84633a&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=2e1db0cf27684f389b9320673721684d => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:48.512910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074815b8-f37e-48a0-9e5c-d244a3d2da5d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:48.618307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482ba02-17f5-432c-a2d3-bb88e50e2ebd None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:48.620166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:48.630848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:48.660617 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074815b8-f37e-48a0-9e5c-d244a3d2da5d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:48.674155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ce9d0-e3b8-49ca-84b1-e0c6857cb9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:48.675151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ce9d0-e3b8-49ca-84b1-e0c6857cb9c2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.675437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ce9d0-e3b8-49ca-84b1-e0c6857cb9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.675718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ce9d0-e3b8-49ca-84b1-e0c6857cb9c2 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:48.676495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ce9d0-e3b8-49ca-84b1-e0c6857cb9c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:48.676801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ce9d0-e3b8-49ca-84b1-e0c6857cb9c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:48.679061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ce9d0-e3b8-49ca-84b1-e0c6857cb9c2 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:48.691433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 182/364] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:48.692843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8046c-0504-4fd7-8554-303d2a121fd5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.693138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8046c-0504-4fd7-8554-303d2a121fd5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.693417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8046c-0504-4fd7-8554-303d2a121fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:48.716074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-acc8046c-0504-4fd7-8554-303d2a121fd5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:48.716660 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e5a90-d669-4be5-8364-8c9d76b8a7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:48.717298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-acc8046c-0504-4fd7-8554-303d2a121fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:48.717559 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e5a90-d669-4be5-8364-8c9d76b8a7a1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.719314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e5a90-d669-4be5-8364-8c9d76b8a7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.719314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e5a90-d669-4be5-8364-8c9d76b8a7a1 None admin] PATH_INFO: `/v3/users/8c9fc8762f4d4652861ba0a3ef1c7c60` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:48.719958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e5a90-d669-4be5-8364-8c9d76b8a7a1 None admin] RBAC: Authorizing `identity:get_user(user_id=8c9fc8762f4d4652861ba0a3ef1c7c60)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:48.722143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e5a90-d669-4be5-8364-8c9d76b8a7a1 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:48.724129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 183/365] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:39:48 2022] GET /identity/v3/users/8c9fc8762f4d4652861ba0a3ef1c7c60 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:48.724970 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e5a90-d669-4be5-8364-8c9d76b8a7a1 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:48.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 183/366] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:48.804202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8046c-0504-4fd7-8554-303d2a121fd5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:48.869116 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e5a90-d669-4be5-8364-8c9d76b8a7a1 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:48.916130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72d29e-64ca-4d55-903d-6680aa228f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:48.917380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72d29e-64ca-4d55-903d-6680aa228f0f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.917891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72d29e-64ca-4d55-903d-6680aa228f0f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.918368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72d29e-64ca-4d55-903d-6680aa228f0f None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:48.919771 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72d29e-64ca-4d55-903d-6680aa228f0f None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:48.921419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72d29e-64ca-4d55-903d-6680aa228f0f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:48.925066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 184/367] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:48 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:48.926965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72d29e-64ca-4d55-903d-6680aa228f0f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:48.940004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:48.946120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:48.953399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8046c-0504-4fd7-8554-303d2a121fd5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:48.992047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbe4fb-7671-424b-a8e9-4b350a144e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:48.992706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbe4fb-7671-424b-a8e9-4b350a144e96 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:48.992984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbe4fb-7671-424b-a8e9-4b350a144e96 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:48.993256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbe4fb-7671-424b-a8e9-4b350a144e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:49.037355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72d29e-64ca-4d55-903d-6680aa228f0f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:49.069866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbe4fb-7671-424b-a8e9-4b350a144e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:49.071136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbe4fb-7671-424b-a8e9-4b350a144e96 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:49.073718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ec813-501e-4f85-aa9d-448b6b7eabcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:49.074614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec813-501e-4f85-aa9d-448b6b7eabcc None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:49.075084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec813-501e-4f85-aa9d-448b6b7eabcc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:49.075555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ec813-501e-4f85-aa9d-448b6b7eabcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:49.076593 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ec813-501e-4f85-aa9d-448b6b7eabcc None admin] RBAC: Adding query filter params (role.id=0c5069e08baa4820b7154564a801ba7e, user.id=8c9fc8762f4d4652861ba0a3ef1c7c60, scope.system=all) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:49.077121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ec813-501e-4f85-aa9d-448b6b7eabcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:49.078778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ec813-501e-4f85-aa9d-448b6b7eabcc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:49.092290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 185/368] 10.222.0.66 () {56 vars in 1308 bytes} [Fri Jul 1 19:39:48 2022] GET /identity/v3/role_assignments?user.id=8c9fc8762f4d4652861ba0a3ef1c7c60&scope.system=all&role.id=0c5069e08baa4820b7154564a801ba7e => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:49.093388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ec813-501e-4f85-aa9d-448b6b7eabcc None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:49.164432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:49.203964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ec813-501e-4f85-aa9d-448b6b7eabcc None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:49.206503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:49.217126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 184/369] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:49.218025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbe4fb-7671-424b-a8e9-4b350a144e96 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:49.244822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fa3d7-b64b-441c-95d5-3cff2d603e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:49.246048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fa3d7-b64b-441c-95d5-3cff2d603e2f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:49.246525 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fa3d7-b64b-441c-95d5-3cff2d603e2f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:49.246987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fa3d7-b64b-441c-95d5-3cff2d603e2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:49.259426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fa3d7-b64b-441c-95d5-3cff2d603e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-d95fa3d7-b64b-441c-95d5-3cff2d603e2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:49.267857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:49.268845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:49.269698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:49.272511 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 186/370] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:49.277602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fa3d7-b64b-441c-95d5-3cff2d603e2f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:49.333875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbe4fb-7671-424b-a8e9-4b350a144e96 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:49.374948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f12d0-3abc-43bf-b3ce-5d380e5e43ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:49.375814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f12d0-3abc-43bf-b3ce-5d380e5e43ac None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:49.376092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f12d0-3abc-43bf-b3ce-5d380e5e43ac None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:49.376403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f12d0-3abc-43bf-b3ce-5d380e5e43ac None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:49.377820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f12d0-3abc-43bf-b3ce-5d380e5e43ac None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:49.379407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f12d0-3abc-43bf-b3ce-5d380e5e43ac None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:49.381126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 185/371] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:49 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:49.381942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f12d0-3abc-43bf-b3ce-5d380e5e43ac None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:49.393055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fa3d7-b64b-441c-95d5-3cff2d603e2f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:49.422347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:49.433000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09599ac-655c-4596-94e8-56d4705d172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:49.434495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09599ac-655c-4596-94e8-56d4705d172e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:49.434992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09599ac-655c-4596-94e8-56d4705d172e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:49.435477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09599ac-655c-4596-94e8-56d4705d172e None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:49.436935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09599ac-655c-4596-94e8-56d4705d172e None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:49.438658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09599ac-655c-4596-94e8-56d4705d172e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:49.441698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 187/372] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:49 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:49.442709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09599ac-655c-4596-94e8-56d4705d172e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:49.470630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:49.511573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f12d0-3abc-43bf-b3ce-5d380e5e43ac None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:49.551931 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ede43a-48b0-4b38-a993-f5557121de1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:49.552613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ede43a-48b0-4b38-a993-f5557121de1e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:49.552894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ede43a-48b0-4b38-a993-f5557121de1e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:49.553166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ede43a-48b0-4b38-a993-f5557121de1e None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:49.553966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ede43a-48b0-4b38-a993-f5557121de1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:49.554485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ede43a-48b0-4b38-a993-f5557121de1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:49.555858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ede43a-48b0-4b38-a993-f5557121de1e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:49.566942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 186/373] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:49.567638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ede43a-48b0-4b38-a993-f5557121de1e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:49.577123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09599ac-655c-4596-94e8-56d4705d172e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:49.616364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4bf38-3985-4f49-8e1c-dcfe483fea2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:49.617407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4bf38-3985-4f49-8e1c-dcfe483fea2a None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:49.617912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4bf38-3985-4f49-8e1c-dcfe483fea2a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:49.618382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4bf38-3985-4f49-8e1c-dcfe483fea2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:49.619412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4bf38-3985-4f49-8e1c-dcfe483fea2a None admin] RBAC: Adding query filter params (group.id=8e1642a69c37473e8e4d86777f84633a, role.id=2e1db0cf27684f389b9320673721684d, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:49.619927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4bf38-3985-4f49-8e1c-dcfe483fea2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:49.621564 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4bf38-3985-4f49-8e1c-dcfe483fea2a None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:49.627597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 188/374] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 1 19:39:49 2022] GET /identity/v3/role_assignments?group.id=8e1642a69c37473e8e4d86777f84633a&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=2e1db0cf27684f389b9320673721684d => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:49.628645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4bf38-3985-4f49-8e1c-dcfe483fea2a None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:49.696462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:49.705080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ede43a-48b0-4b38-a993-f5557121de1e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:49.742452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:49.748531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4bf38-3985-4f49-8e1c-dcfe483fea2a None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:49.759935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09845272-c6f4-4995-b1a8-7792e0f9cda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:49.760869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09845272-c6f4-4995-b1a8-7792e0f9cda7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:49.761151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09845272-c6f4-4995-b1a8-7792e0f9cda7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:49.761423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09845272-c6f4-4995-b1a8-7792e0f9cda7 None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:49.766671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09845272-c6f4-4995-b1a8-7792e0f9cda7 None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:49.769212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09845272-c6f4-4995-b1a8-7792e0f9cda7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:49.771355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 187/375] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:49 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:49.777167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09845272-c6f4-4995-b1a8-7792e0f9cda7 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:49.790978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f2091-5f1d-4961-9b90-ca5c1ad46238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:49.791996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2091-5f1d-4961-9b90-ca5c1ad46238 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:49.792499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2091-5f1d-4961-9b90-ca5c1ad46238 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:49.792973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f2091-5f1d-4961-9b90-ca5c1ad46238 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:49.908504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f2091-5f1d-4961-9b90-ca5c1ad46238 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:49.910163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f2091-5f1d-4961-9b90-ca5c1ad46238 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:49.991868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09845272-c6f4-4995-b1a8-7792e0f9cda7 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:50.009973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:50.053465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 189/376] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:50.065323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9328-bb19-46b1-90c4-5e66fae1aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:50.065754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f2091-5f1d-4961-9b90-ca5c1ad46238 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:50.066033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9328-bb19-46b1-90c4-5e66fae1aabd None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:50.066314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9328-bb19-46b1-90c4-5e66fae1aabd None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:50.066590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9328-bb19-46b1-90c4-5e66fae1aabd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:50.067439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9328-bb19-46b1-90c4-5e66fae1aabd None admin] RBAC: Adding query filter params (group.id=8e1642a69c37473e8e4d86777f84633a, role.id=b127031d393048d7ae9a35015113e82a, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:50.067747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9328-bb19-46b1-90c4-5e66fae1aabd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:50.069194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9328-bb19-46b1-90c4-5e66fae1aabd None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:50.076503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 188/377] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 1 19:39:49 2022] GET /identity/v3/role_assignments?group.id=8e1642a69c37473e8e4d86777f84633a&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=b127031d393048d7ae9a35015113e82a => generated 260 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:50.211082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f2091-5f1d-4961-9b90-ca5c1ad46238 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:50.221044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1b7a8-20ed-4aeb-bdd4-2aa2d451a154 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:50.221662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1b7a8-20ed-4aeb-bdd4-2aa2d451a154 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:50.222044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1b7a8-20ed-4aeb-bdd4-2aa2d451a154 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:50.223570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 189/378] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:50.230016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22be28-564b-4b65-bec3-07f48e464bce None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:50.230401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22be28-564b-4b65-bec3-07f48e464bce None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:50.230777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22be28-564b-4b65-bec3-07f48e464bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:50.249986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281556f-b1e6-408a-b377-830800608a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:50.250782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281556f-b1e6-408a-b377-830800608a9a None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:50.251060 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281556f-b1e6-408a-b377-830800608a9a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:50.251872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281556f-b1e6-408a-b377-830800608a9a None admin] PATH_INFO: `/v3/projects/54a2d5d499f64a6a8437120d4d4f1f46` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:50.253085 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281556f-b1e6-408a-b377-830800608a9a None admin] RBAC: Authorizing `identity:get_project(project_id=54a2d5d499f64a6a8437120d4d4f1f46)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:50.253868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-3e22be28-564b-4b65-bec3-07f48e464bce None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:50.254600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281556f-b1e6-408a-b377-830800608a9a None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:50.255271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3e22be28-564b-4b65-bec3-07f48e464bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:50.256621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 190/379] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:50 2022] GET /identity/v3/projects/54a2d5d499f64a6a8437120d4d4f1f46 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:50.261738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281556f-b1e6-408a-b377-830800608a9a None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:50.293943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:50.317975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:50.338502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 190/380] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:50.364238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22be28-564b-4b65-bec3-07f48e464bce None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:50.377835 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281556f-b1e6-408a-b377-830800608a9a None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:50.415562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbaf8c-e6cf-459f-ae27-d16f9b163d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:50.416497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbaf8c-e6cf-459f-ae27-d16f9b163d1c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:50.416774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbaf8c-e6cf-459f-ae27-d16f9b163d1c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:50.417097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbaf8c-e6cf-459f-ae27-d16f9b163d1c None admin] PATH_INFO: `/v3/groups/3fb9745e730b470aadf5ea697e5748e5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:50.422741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbaf8c-e6cf-459f-ae27-d16f9b163d1c None admin] RBAC: Authorizing `identity:get_group(group_id=3fb9745e730b470aadf5ea697e5748e5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:50.424392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbaf8c-e6cf-459f-ae27-d16f9b163d1c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:50.426655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 191/381] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:50 2022] GET /identity/v3/groups/3fb9745e730b470aadf5ea697e5748e5 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:50.432831 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbaf8c-e6cf-459f-ae27-d16f9b163d1c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:50.477887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22be28-564b-4b65-bec3-07f48e464bce None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:50.519998 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f14460-053a-498e-8ec9-144501665e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:50.521017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14460-053a-498e-8ec9-144501665e42 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:50.521933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14460-053a-498e-8ec9-144501665e42 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:50.522220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14460-053a-498e-8ec9-144501665e42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:50.537049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f14460-053a-498e-8ec9-144501665e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-86f14460-053a-498e-8ec9-144501665e42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:50.550919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:50.551830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:50.552598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 191/382] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:39:50.558570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f14460-053a-498e-8ec9-144501665e42 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:50.559541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:50.575000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbaf8c-e6cf-459f-ae27-d16f9b163d1c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:50.625951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:50.632078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9aea7c-d527-41cd-b4ec-4d2937027e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:50.633279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aea7c-d527-41cd-b4ec-4d2937027e39 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:50.633813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aea7c-d527-41cd-b4ec-4d2937027e39 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:50.634271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aea7c-d527-41cd-b4ec-4d2937027e39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:50.635290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9aea7c-d527-41cd-b4ec-4d2937027e39 None admin] RBAC: Adding query filter params (group.id=3fb9745e730b470aadf5ea697e5748e5, role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=54a2d5d499f64a6a8437120d4d4f1f46) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:50.636239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9aea7c-d527-41cd-b4ec-4d2937027e39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:50.638180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9aea7c-d527-41cd-b4ec-4d2937027e39 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:50.649636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 192/383] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 1 19:39:50 2022] GET /identity/v3/role_assignments?group.id=3fb9745e730b470aadf5ea697e5748e5&scope.project.id=54a2d5d499f64a6a8437120d4d4f1f46&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:50.703321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f14460-053a-498e-8ec9-144501665e42 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:50.744008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88183f3-48e2-4c77-8528-9dbb739f8ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:50.745399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88183f3-48e2-4c77-8528-9dbb739f8ba9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:50.745881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88183f3-48e2-4c77-8528-9dbb739f8ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:50.746301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88183f3-48e2-4c77-8528-9dbb739f8ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:50.747239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88183f3-48e2-4c77-8528-9dbb739f8ba9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:50.747714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88183f3-48e2-4c77-8528-9dbb739f8ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:50.749271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88183f3-48e2-4c77-8528-9dbb739f8ba9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:50.760482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 192/384] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:50.766337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9aea7c-d527-41cd-b4ec-4d2937027e39 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:50.889312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9aea7c-d527-41cd-b4ec-4d2937027e39 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:50.928036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1154c0-eace-4ea5-ad61-5850653e09a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:50.929161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1154c0-eace-4ea5-ad61-5850653e09a7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:50.929617 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1154c0-eace-4ea5-ad61-5850653e09a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:50.930048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1154c0-eace-4ea5-ad61-5850653e09a7 None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:50.931458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1154c0-eace-4ea5-ad61-5850653e09a7 None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:50.933055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1154c0-eace-4ea5-ad61-5850653e09a7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:50.935175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 193/385] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:50 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:50.940958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88183f3-48e2-4c77-8528-9dbb739f8ba9 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:51.056080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88183f3-48e2-4c77-8528-9dbb739f8ba9 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:51.066631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:51.094017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac31263-8a74-4e74-9169-04dab8540ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:51.095210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac31263-8a74-4e74-9169-04dab8540ef1 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:51.095635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac31263-8a74-4e74-9169-04dab8540ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:51.096073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac31263-8a74-4e74-9169-04dab8540ef1 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:51.097434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac31263-8a74-4e74-9169-04dab8540ef1 None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:51.099056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac31263-8a74-4e74-9169-04dab8540ef1 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:51.101199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 193/386] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:50 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:51.106654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1154c0-eace-4ea5-ad61-5850653e09a7 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:51.136016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:51.224960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1154c0-eace-4ea5-ad61-5850653e09a7 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:51.269098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1f3c0-d579-4d70-8b9e-c053684c1543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:51.270270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1f3c0-d579-4d70-8b9e-c053684c1543 None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:51.270705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1f3c0-d579-4d70-8b9e-c053684c1543 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:51.271142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1f3c0-d579-4d70-8b9e-c053684c1543 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460/groups/8e1642a69c37473e8e4d86777f84633a/roles/b127031d393048d7ae9a35015113e82a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:51.273607 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1f3c0-d579-4d70-8b9e-c053684c1543 None admin] RBAC: Authorizing `identity:create_grant(project_id=574afa1590854c65a807df4589e2c460, group_id=8e1642a69c37473e8e4d86777f84633a, role_id=b127031d393048d7ae9a35015113e82a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:51.275304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1f3c0-d579-4d70-8b9e-c053684c1543 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:51.304302 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 194/387] 10.222.0.66 () {58 vars in 1442 bytes} [Fri Jul 1 19:39:51 2022] PUT /identity/v3/projects/574afa1590854c65a807df4589e2c460/groups/8e1642a69c37473e8e4d86777f84633a/roles/b127031d393048d7ae9a35015113e82a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:51.578443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beb5dd-a558-4d08-aa46-119a473e4588 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:51.578772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beb5dd-a558-4d08-aa46-119a473e4588 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:51.578988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beb5dd-a558-4d08-aa46-119a473e4588 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:51.581268 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 194/388] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:51.587016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf83c11-bc15-4cf2-af92-e5486bf85a77 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:51.587304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf83c11-bc15-4cf2-af92-e5486bf85a77 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:51.587586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf83c11-bc15-4cf2-af92-e5486bf85a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:51.608195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-aaf83c11-bc15-4cf2-af92-e5486bf85a77 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:51.610025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-aaf83c11-bc15-4cf2-af92-e5486bf85a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:51.692772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 195/389] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:51.715167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beb5dd-a558-4d08-aa46-119a473e4588 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:51.824223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beb5dd-a558-4d08-aa46-119a473e4588 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:51.859710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d998-6ed0-45b5-b2cc-2704cd5d8053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:51.860656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d998-6ed0-45b5-b2cc-2704cd5d8053 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:51.860938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d998-6ed0-45b5-b2cc-2704cd5d8053 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:51.861205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d998-6ed0-45b5-b2cc-2704cd5d8053 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:51.871921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55d998-6ed0-45b5-b2cc-2704cd5d8053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-1b55d998-6ed0-45b5-b2cc-2704cd5d8053 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:51.880595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:51.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:51.882287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:51.889610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 195/390] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:51.889749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf83c11-bc15-4cf2-af92-e5486bf85a77 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:51.905477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:52.020007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf83c11-bc15-4cf2-af92-e5486bf85a77 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:52.066964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f5e45-6d13-4db3-9246-d31335804c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:52.067641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f5e45-6d13-4db3-9246-d31335804c9e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:52.067967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f5e45-6d13-4db3-9246-d31335804c9e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:52.068243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f5e45-6d13-4db3-9246-d31335804c9e None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:52.069001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f5e45-6d13-4db3-9246-d31335804c9e None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:52.069304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f5e45-6d13-4db3-9246-d31335804c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:52.070709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f5e45-6d13-4db3-9246-d31335804c9e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:52.080500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 196/391] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:52.085977 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d998-6ed0-45b5-b2cc-2704cd5d8053 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:52.129926 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:52.206334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d998-6ed0-45b5-b2cc-2704cd5d8053 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:52.250304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721917a6-1859-43db-b41c-2f54cc4f532f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:52.251261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721917a6-1859-43db-b41c-2f54cc4f532f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:52.252054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721917a6-1859-43db-b41c-2f54cc4f532f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:52.252054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721917a6-1859-43db-b41c-2f54cc4f532f None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:52.258361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721917a6-1859-43db-b41c-2f54cc4f532f None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:52.259835 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721917a6-1859-43db-b41c-2f54cc4f532f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:52.261567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 196/392] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:52 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:52.268036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f5e45-6d13-4db3-9246-d31335804c9e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:52.334681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb3bb8-7435-4faa-bcae-d560d4332ab9 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:52.334868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb3bb8-7435-4faa-bcae-d560d4332ab9 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:52.335143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb3bb8-7435-4faa-bcae-d560d4332ab9 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:52.336514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 197/393] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:39:52.345626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01444df1-7874-4e84-a88e-befb1fd368c3 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:52.345626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01444df1-7874-4e84-a88e-befb1fd368c3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:52.345626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01444df1-7874-4e84-a88e-befb1fd368c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:52.377229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-01444df1-7874-4e84-a88e-befb1fd368c3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:52.378822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-01444df1-7874-4e84-a88e-befb1fd368c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:52.381331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:52.414251 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f5e45-6d13-4db3-9246-d31335804c9e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:52.448713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d61f4f-bce1-4b8a-ba32-3a3739471f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:52.449977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d61f4f-bce1-4b8a-ba32-3a3739471f7c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:52.450463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d61f4f-bce1-4b8a-ba32-3a3739471f7c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:52.450947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d61f4f-bce1-4b8a-ba32-3a3739471f7c None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:52.452361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d61f4f-bce1-4b8a-ba32-3a3739471f7c None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:52.454063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d61f4f-bce1-4b8a-ba32-3a3739471f7c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:52.457174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 197/394] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:52 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:52.462969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d61f4f-bce1-4b8a-ba32-3a3739471f7c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:52.532838 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 198/395] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:52.571239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d61f4f-bce1-4b8a-ba32-3a3739471f7c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:52.574955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01444df1-7874-4e84-a88e-befb1fd368c3 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:52.607797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef422e1-27b0-4fe9-bd51-12f02af754d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:52.608844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef422e1-27b0-4fe9-bd51-12f02af754d7 None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:52.609122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef422e1-27b0-4fe9-bd51-12f02af754d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:52.609396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef422e1-27b0-4fe9-bd51-12f02af754d7 None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460/groups/8e1642a69c37473e8e4d86777f84633a/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:52.616263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef422e1-27b0-4fe9-bd51-12f02af754d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=574afa1590854c65a807df4589e2c460, group_id=8e1642a69c37473e8e4d86777f84633a, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:52.617753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef422e1-27b0-4fe9-bd51-12f02af754d7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:52.635545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:52.647379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 198/396] 10.222.0.66 () {58 vars in 1442 bytes} [Fri Jul 1 19:39:52 2022] PUT /identity/v3/projects/574afa1590854c65a807df4589e2c460/groups/8e1642a69c37473e8e4d86777f84633a/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:39:52.732396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01444df1-7874-4e84-a88e-befb1fd368c3 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:52.779558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec9c0-5a84-48bc-bbc0-d4a580f50d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:52.780352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec9c0-5a84-48bc-bbc0-d4a580f50d32 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:52.780630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec9c0-5a84-48bc-bbc0-d4a580f50d32 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:52.780897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec9c0-5a84-48bc-bbc0-d4a580f50d32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:52.794155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ec9c0-5a84-48bc-bbc0-d4a580f50d32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-cc3ec9c0-5a84-48bc-bbc0-d4a580f50d32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:52.806940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:52.807985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:52.808776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 199/397] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:52.815366 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef422e1-27b0-4fe9-bd51-12f02af754d7 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:52.819962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:52.954483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef422e1-27b0-4fe9-bd51-12f02af754d7 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:52.996152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efa1a1-4aa2-4340-a345-ecc1f789b7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:52.996830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efa1a1-4aa2-4340-a345-ecc1f789b7a7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:52.997109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efa1a1-4aa2-4340-a345-ecc1f789b7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:52.997383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efa1a1-4aa2-4340-a345-ecc1f789b7a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:52.998170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efa1a1-4aa2-4340-a345-ecc1f789b7a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:52.998473 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efa1a1-4aa2-4340-a345-ecc1f789b7a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:52.999861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efa1a1-4aa2-4340-a345-ecc1f789b7a7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:53.009686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 199/398] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:53.015287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec9c0-5a84-48bc-bbc0-d4a580f50d32 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:53.109936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b7958-9c69-4ea3-8341-2fd193e95830 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:53.110271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b7958-9c69-4ea3-8341-2fd193e95830 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:53.111024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b7958-9c69-4ea3-8341-2fd193e95830 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:53.112555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 200/399] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:53.121533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4b4b7-f0d8-44e2-af2f-1cc9e68aad72 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:53.121533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4b4b7-f0d8-44e2-af2f-1cc9e68aad72 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:53.121533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4b4b7-f0d8-44e2-af2f-1cc9e68aad72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:53.142567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-1ff4b4b7-f0d8-44e2-af2f-1cc9e68aad72 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:53.145243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-1ff4b4b7-f0d8-44e2-af2f-1cc9e68aad72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:53.165945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:53.174256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec9c0-5a84-48bc-bbc0-d4a580f50d32 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:53.231337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3161efd-a9e4-4347-b842-72660e799ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:53.232108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3161efd-a9e4-4347-b842-72660e799ea2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:53.232383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3161efd-a9e4-4347-b842-72660e799ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:53.232650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3161efd-a9e4-4347-b842-72660e799ea2 None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:53.234131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3161efd-a9e4-4347-b842-72660e799ea2 None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:53.236138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3161efd-a9e4-4347-b842-72660e799ea2 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:53.238456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 200/400] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:53 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:53.246484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3161efd-a9e4-4347-b842-72660e799ea2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:53.265622 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:53.267900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 201/401] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:53.346488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4b4b7-f0d8-44e2-af2f-1cc9e68aad72 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:53.426604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3161efd-a9e4-4347-b842-72660e799ea2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:53.463155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88b12c-b397-490b-b3f9-0f737168425e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:53.464424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88b12c-b397-490b-b3f9-0f737168425e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:53.464858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88b12c-b397-490b-b3f9-0f737168425e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:53.465284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88b12c-b397-490b-b3f9-0f737168425e None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:53.466830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88b12c-b397-490b-b3f9-0f737168425e None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:53.468411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88b12c-b397-490b-b3f9-0f737168425e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:53.471342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 201/402] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:53 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:53.472724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a716e6-b0d5-4385-b08c-162fdd9aae55 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:53.473164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a716e6-b0d5-4385-b08c-162fdd9aae55 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:53.473612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a716e6-b0d5-4385-b08c-162fdd9aae55 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:53.479050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 202/403] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:53.479978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a716e6-b0d5-4385-b08c-162fdd9aae55 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:53.496583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4b4b7-f0d8-44e2-af2f-1cc9e68aad72 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:53.519406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:53.540977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f351c4-811f-40f3-b05a-79c55be736dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:53.541866 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f351c4-811f-40f3-b05a-79c55be736dc None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:53.542143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f351c4-811f-40f3-b05a-79c55be736dc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:53.542415 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f351c4-811f-40f3-b05a-79c55be736dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:53.561650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f351c4-811f-40f3-b05a-79c55be736dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-c2f351c4-811f-40f3-b05a-79c55be736dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:53.567659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:53.568532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:53.569295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:53.569926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 202/404] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:53.573684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c50fd-25ee-4f83-ad5b-c537e849fc89 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:53.573684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c50fd-25ee-4f83-ad5b-c537e849fc89 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:53.573684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c50fd-25ee-4f83-ad5b-c537e849fc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:53.592918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-e46c50fd-25ee-4f83-ad5b-c537e849fc89 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:53.594669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e46c50fd-25ee-4f83-ad5b-c537e849fc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:53.619327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a716e6-b0d5-4385-b08c-162fdd9aae55 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:53.665155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:53.672639 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a3d84-9707-4ca8-a307-542ceacf4ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:53.674379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a3d84-9707-4ca8-a307-542ceacf4ece None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:53.674677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a3d84-9707-4ca8-a307-542ceacf4ece None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:53.674954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a3d84-9707-4ca8-a307-542ceacf4ece None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/groups/8e1642a69c37473e8e4d86777f84633a/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:53.677898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a3d84-9707-4ca8-a307-542ceacf4ece None admin] RBAC: Authorizing `identity:create_grant(project_id=3d0bca94154a400bb0d7c93638e9b20f, group_id=8e1642a69c37473e8e4d86777f84633a, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:53.679861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a3d84-9707-4ca8-a307-542ceacf4ece None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:53.685287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 203/405] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:53.686282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c50fd-25ee-4f83-ad5b-c537e849fc89 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:53.716055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 203/406] 10.222.0.66 () {58 vars in 1442 bytes} [Fri Jul 1 19:39:53 2022] PUT /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/groups/8e1642a69c37473e8e4d86777f84633a/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:53.718117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a3d84-9707-4ca8-a307-542ceacf4ece None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:53.870305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:53.873124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c50fd-25ee-4f83-ad5b-c537e849fc89 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:53.897757 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a3d84-9707-4ca8-a307-542ceacf4ece None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:53.941995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e35ad-0ecc-4ab3-ae34-aced87ec2004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:53.942746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e35ad-0ecc-4ab3-ae34-aced87ec2004 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:53.943024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e35ad-0ecc-4ab3-ae34-aced87ec2004 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:53.943346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e35ad-0ecc-4ab3-ae34-aced87ec2004 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:53.944111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e35ad-0ecc-4ab3-ae34-aced87ec2004 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:53.944749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e35ad-0ecc-4ab3-ae34-aced87ec2004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:53.946166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e35ad-0ecc-4ab3-ae34-aced87ec2004 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:53.958677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 204/407] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:53.965141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e35ad-0ecc-4ab3-ae34-aced87ec2004 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:53.970652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e86967-4903-4dac-a22e-6d4eeff5a2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:53.971509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e86967-4903-4dac-a22e-6d4eeff5a2b8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:53.971786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e86967-4903-4dac-a22e-6d4eeff5a2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:53.972225 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e86967-4903-4dac-a22e-6d4eeff5a2b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:53.984218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e86967-4903-4dac-a22e-6d4eeff5a2b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-74e86967-4903-4dac-a22e-6d4eeff5a2b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:53.993516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:53.994671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:53.995651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:53.998034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 204/408] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:39:54.004162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e86967-4903-4dac-a22e-6d4eeff5a2b8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:54.018034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:54.083223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e35ad-0ecc-4ab3-ae34-aced87ec2004 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:54.123545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364829ee-5c4e-4533-bc62-8ef9a145697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:54.124334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364829ee-5c4e-4533-bc62-8ef9a145697a None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.124615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364829ee-5c4e-4533-bc62-8ef9a145697a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.124892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364829ee-5c4e-4533-bc62-8ef9a145697a None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.126783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364829ee-5c4e-4533-bc62-8ef9a145697a None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:54.128317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364829ee-5c4e-4533-bc62-8ef9a145697a None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:54.130512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 205/409] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:53 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:54.131826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b6765-84c3-4e9e-b6a9-cdba710746ca None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.132114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b6765-84c3-4e9e-b6a9-cdba710746ca None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.132389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b6765-84c3-4e9e-b6a9-cdba710746ca None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.133894 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 206/410] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:54.138680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b6765-84c3-4e9e-b6a9-cdba710746ca None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:54.140383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e86967-4903-4dac-a22e-6d4eeff5a2b8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:54.185103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538d91e-a2b3-4542-8f0e-1371a64624e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:54.186053 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538d91e-a2b3-4542-8f0e-1371a64624e0 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.186604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538d91e-a2b3-4542-8f0e-1371a64624e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.187539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538d91e-a2b3-4542-8f0e-1371a64624e0 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.188448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538d91e-a2b3-4542-8f0e-1371a64624e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:54.189902 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538d91e-a2b3-4542-8f0e-1371a64624e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:54.192134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538d91e-a2b3-4542-8f0e-1371a64624e0 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:54.208696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 205/411] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:54.208696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb809b-ec23-4a68-a240-e84841abda30 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.208696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb809b-ec23-4a68-a240-e84841abda30 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.208696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb809b-ec23-4a68-a240-e84841abda30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.230856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-5afb809b-ec23-4a68-a240-e84841abda30 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:54.232862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5afb809b-ec23-4a68-a240-e84841abda30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:54.246166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:54.266431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b6765-84c3-4e9e-b6a9-cdba710746ca None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:54.283212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:54.307165 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 206/412] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:54.308096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb809b-ec23-4a68-a240-e84841abda30 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:54.314948 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22b779-96a4-4ef9-8917-a930af3f6d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:54.315909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22b779-96a4-4ef9-8917-a930af3f6d9c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.316269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22b779-96a4-4ef9-8917-a930af3f6d9c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.316617 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22b779-96a4-4ef9-8917-a930af3f6d9c None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.318030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22b779-96a4-4ef9-8917-a930af3f6d9c None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:54.319691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22b779-96a4-4ef9-8917-a930af3f6d9c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:54.326102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 207/413] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:54 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:54.327219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22b779-96a4-4ef9-8917-a930af3f6d9c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:54.421855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb809b-ec23-4a68-a240-e84841abda30 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:54.447295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22b779-96a4-4ef9-8917-a930af3f6d9c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:54.456015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333afcc-f01f-4d3b-afd7-b622f4a2b376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:54.456950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333afcc-f01f-4d3b-afd7-b622f4a2b376 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.457298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333afcc-f01f-4d3b-afd7-b622f4a2b376 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.457662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333afcc-f01f-4d3b-afd7-b622f4a2b376 None admin] PATH_INFO: `/v3/groups/3fb9745e730b470aadf5ea697e5748e5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.458904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333afcc-f01f-4d3b-afd7-b622f4a2b376 None admin] RBAC: Authorizing `identity:get_group(group_id=3fb9745e730b470aadf5ea697e5748e5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:54.460394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333afcc-f01f-4d3b-afd7-b622f4a2b376 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:54.462420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 207/414] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:54 2022] GET /identity/v3/groups/3fb9745e730b470aadf5ea697e5748e5 => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:54.463289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333afcc-f01f-4d3b-afd7-b622f4a2b376 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:54.475828 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:54.488254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22977e82-80d3-459a-b629-41c6f0918978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:54.489309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22977e82-80d3-459a-b629-41c6f0918978 None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.489741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22977e82-80d3-459a-b629-41c6f0918978 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.490021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22977e82-80d3-459a-b629-41c6f0918978 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/groups/8e1642a69c37473e8e4d86777f84633a/roles/b127031d393048d7ae9a35015113e82a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.492545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22977e82-80d3-459a-b629-41c6f0918978 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d0bca94154a400bb0d7c93638e9b20f, group_id=8e1642a69c37473e8e4d86777f84633a, role_id=b127031d393048d7ae9a35015113e82a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:54.494033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22977e82-80d3-459a-b629-41c6f0918978 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:54.511874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 208/415] 10.222.0.66 () {58 vars in 1442 bytes} [Fri Jul 1 19:39:54 2022] PUT /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f/groups/8e1642a69c37473e8e4d86777f84633a/roles/b127031d393048d7ae9a35015113e82a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:39:54.512904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22977e82-80d3-459a-b629-41c6f0918978 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:54.570501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:54.576036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333afcc-f01f-4d3b-afd7-b622f4a2b376 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:54.614148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea559d4d-d75c-439d-8d3d-683fa3a8d775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:54.614962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea559d4d-d75c-439d-8d3d-683fa3a8d775 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.615237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea559d4d-d75c-439d-8d3d-683fa3a8d775 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.615504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea559d4d-d75c-439d-8d3d-683fa3a8d775 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.626809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea559d4d-d75c-439d-8d3d-683fa3a8d775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-ea559d4d-d75c-439d-8d3d-683fa3a8d775 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:54.635206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:54.637358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:54.639399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 208/416] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:54.648838 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22977e82-80d3-459a-b629-41c6f0918978 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:54.650615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea559d4d-d75c-439d-8d3d-683fa3a8d775 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:54.690506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae53e92-281d-4791-b9df-687019a4425b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:54.691381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53e92-281d-4791-b9df-687019a4425b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.691668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53e92-281d-4791-b9df-687019a4425b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.691944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae53e92-281d-4791-b9df-687019a4425b None admin] PATH_INFO: `/v3/projects/54a2d5d499f64a6a8437120d4d4f1f46` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.693278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae53e92-281d-4791-b9df-687019a4425b None admin] RBAC: Authorizing `identity:get_project(project_id=54a2d5d499f64a6a8437120d4d4f1f46)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:54.694839 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae53e92-281d-4791-b9df-687019a4425b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:54.696880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 209/417] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:54 2022] GET /identity/v3/projects/54a2d5d499f64a6a8437120d4d4f1f46 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:54.702788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae53e92-281d-4791-b9df-687019a4425b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:54.769888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:54.818722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea559d4d-d75c-439d-8d3d-683fa3a8d775 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:54.835806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae53e92-281d-4791-b9df-687019a4425b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:54.851355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:54.858959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ac33b-d6e7-4184-9ec7-76c09b1846e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:54.859622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ac33b-d6e7-4184-9ec7-76c09b1846e3 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.860007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ac33b-d6e7-4184-9ec7-76c09b1846e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.860284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ac33b-d6e7-4184-9ec7-76c09b1846e3 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.861513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ac33b-d6e7-4184-9ec7-76c09b1846e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:54.861859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ac33b-d6e7-4184-9ec7-76c09b1846e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:54.863235 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ac33b-d6e7-4184-9ec7-76c09b1846e3 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:54.877455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 209/418] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:54.879118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ac33b-d6e7-4184-9ec7-76c09b1846e3 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:54.889133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca308d8b-ed33-4f5d-adec-3e34911dce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:54.890333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca308d8b-ed33-4f5d-adec-3e34911dce79 None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:54.891259 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca308d8b-ed33-4f5d-adec-3e34911dce79 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:54.892346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca308d8b-ed33-4f5d-adec-3e34911dce79 None admin] PATH_INFO: `/v3/projects/54a2d5d499f64a6a8437120d4d4f1f46/groups/3fb9745e730b470aadf5ea697e5748e5/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:54.895700 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca308d8b-ed33-4f5d-adec-3e34911dce79 None admin] RBAC: Authorizing `identity:create_grant(project_id=54a2d5d499f64a6a8437120d4d4f1f46, group_id=3fb9745e730b470aadf5ea697e5748e5, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:54.898365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca308d8b-ed33-4f5d-adec-3e34911dce79 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:54.917902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 210/419] 10.222.0.66 () {58 vars in 1442 bytes} [Fri Jul 1 19:39:54 2022] PUT /identity/v3/projects/54a2d5d499f64a6a8437120d4d4f1f46/groups/3fb9745e730b470aadf5ea697e5748e5/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:39:55.049868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ac33b-d6e7-4184-9ec7-76c09b1846e3 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:55.090429 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:55.099767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67adb0-2cc8-4f18-a3bc-177af3629cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:55.100487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67adb0-2cc8-4f18-a3bc-177af3629cea None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:55.100772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67adb0-2cc8-4f18-a3bc-177af3629cea None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:55.101104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67adb0-2cc8-4f18-a3bc-177af3629cea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:55.183294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67adb0-2cc8-4f18-a3bc-177af3629cea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:55.184758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67adb0-2cc8-4f18-a3bc-177af3629cea None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:55.293311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 210/420] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:55.299477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3560ce-bb97-4848-898b-fbda728f4af2 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:55.300010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67adb0-2cc8-4f18-a3bc-177af3629cea None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:55.300528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3560ce-bb97-4848-898b-fbda728f4af2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:55.301029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3560ce-bb97-4848-898b-fbda728f4af2 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:55.304214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 211/421] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:55.309398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc3c9-585f-4a6c-86f0-6d6bcd54531f None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:55.310286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc3c9-585f-4a6c-86f0-6d6bcd54531f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:55.310788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc3c9-585f-4a6c-86f0-6d6bcd54531f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:55.344405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c68bc3c9-585f-4a6c-86f0-6d6bcd54531f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:55.348987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:55.349521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c68bc3c9-585f-4a6c-86f0-6d6bcd54531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:55.461974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 212/422] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:55.486540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc3c9-585f-4a6c-86f0-6d6bcd54531f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:55.503963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67adb0-2cc8-4f18-a3bc-177af3629cea None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:55.564077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52e0e2-d5c7-4c46-913d-1f0f1bf9b43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:55.565088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52e0e2-d5c7-4c46-913d-1f0f1bf9b43f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:55.565487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52e0e2-d5c7-4c46-913d-1f0f1bf9b43f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:55.565877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52e0e2-d5c7-4c46-913d-1f0f1bf9b43f None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:55.567210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52e0e2-d5c7-4c46-913d-1f0f1bf9b43f None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:55.569336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52e0e2-d5c7-4c46-913d-1f0f1bf9b43f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:55.575092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 211/423] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:55 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:55.577737 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52e0e2-d5c7-4c46-913d-1f0f1bf9b43f None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:55.685563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52e0e2-d5c7-4c46-913d-1f0f1bf9b43f None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:55.709798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc3c9-585f-4a6c-86f0-6d6bcd54531f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:55.713889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:55.714027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:55.722900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810078a-778a-490b-a7a7-698e1e2236b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:55.723961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810078a-778a-490b-a7a7-698e1e2236b6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:55.724390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810078a-778a-490b-a7a7-698e1e2236b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:55.724808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810078a-778a-490b-a7a7-698e1e2236b6 None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:55.726262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8810078a-778a-490b-a7a7-698e1e2236b6 None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:55.728097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8810078a-778a-490b-a7a7-698e1e2236b6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:55.736444 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 212/424] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:55 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:55.738310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810078a-778a-490b-a7a7-698e1e2236b6 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:55.834462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9c27f-5e5d-41f1-923c-003030a3c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:55.835266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9c27f-5e5d-41f1-923c-003030a3c761 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:55.835539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9c27f-5e5d-41f1-923c-003030a3c761 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:55.835806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9c27f-5e5d-41f1-923c-003030a3c761 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:55.857952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9c27f-5e5d-41f1-923c-003030a3c761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-82c9c27f-5e5d-41f1-923c-003030a3c761 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:55.866211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:55.867095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:55.867995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:55.876595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810078a-778a-490b-a7a7-698e1e2236b6 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:55.879798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 213/425] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:55.880689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9c27f-5e5d-41f1-923c-003030a3c761 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:55.996475 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9c27f-5e5d-41f1-923c-003030a3c761 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:55.998284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477636ce-412e-4ff9-b72f-21f938e02cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:55.998957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477636ce-412e-4ff9-b72f-21f938e02cf9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:55.999237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477636ce-412e-4ff9-b72f-21f938e02cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:55.999510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477636ce-412e-4ff9-b72f-21f938e02cf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:56.000349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477636ce-412e-4ff9-b72f-21f938e02cf9 None admin] RBAC: Adding query filter params (group.id=8e1642a69c37473e8e4d86777f84633a, role.id=b127031d393048d7ae9a35015113e82a, scope.project.id=574afa1590854c65a807df4589e2c460) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:56.000654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477636ce-412e-4ff9-b72f-21f938e02cf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:56.003290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477636ce-412e-4ff9-b72f-21f938e02cf9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:56.017721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 213/426] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 1 19:39:55 2022] GET /identity/v3/role_assignments?group.id=8e1642a69c37473e8e4d86777f84633a&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=b127031d393048d7ae9a35015113e82a => generated 611 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:56.038257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff339d2-90b1-4d3d-9d8d-ac30a14de1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:56.038958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff339d2-90b1-4d3d-9d8d-ac30a14de1c9 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:56.039570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff339d2-90b1-4d3d-9d8d-ac30a14de1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:56.039570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff339d2-90b1-4d3d-9d8d-ac30a14de1c9 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:56.040262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff339d2-90b1-4d3d-9d8d-ac30a14de1c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:56.040573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff339d2-90b1-4d3d-9d8d-ac30a14de1c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:56.042019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff339d2-90b1-4d3d-9d8d-ac30a14de1c9 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:56.052407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 214/427] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:56.058689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477636ce-412e-4ff9-b72f-21f938e02cf9 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:56.146047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:56.185699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8781-c866-48d2-9638-b2a1c33127d3 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:56.186324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8781-c866-48d2-9638-b2a1c33127d3 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:56.186822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8781-c866-48d2-9638-b2a1c33127d3 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:56.189441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 215/428] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:56.195174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc13e2-6af2-4b65-a245-ce0b3d8608dc None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:56.195782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc13e2-6af2-4b65-a245-ce0b3d8608dc None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:56.196252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc13e2-6af2-4b65-a245-ce0b3d8608dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:56.231236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-8ddc13e2-6af2-4b65-a245-ce0b3d8608dc None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:56.232870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8ddc13e2-6af2-4b65-a245-ce0b3d8608dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:56.260164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:56.263716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477636ce-412e-4ff9-b72f-21f938e02cf9 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:56.308963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f43c33-f1e8-4a26-af07-b67975ece8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:56.310154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f43c33-f1e8-4a26-af07-b67975ece8f3 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:56.310983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f43c33-f1e8-4a26-af07-b67975ece8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:56.311468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f43c33-f1e8-4a26-af07-b67975ece8f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:56.329989 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 216/429] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:56.356615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc13e2-6af2-4b65-a245-ce0b3d8608dc None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:56.398002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f43c33-f1e8-4a26-af07-b67975ece8f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:56.399428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f43c33-f1e8-4a26-af07-b67975ece8f3 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:56.434867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:56.497208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc13e2-6af2-4b65-a245-ce0b3d8608dc None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:56.527211 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 214/430] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:56.529228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f43c33-f1e8-4a26-af07-b67975ece8f3 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:56.540631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae076624-8bd0-4f9d-ae6d-87aafba4c599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:56.541904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae076624-8bd0-4f9d-ae6d-87aafba4c599 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:56.542390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae076624-8bd0-4f9d-ae6d-87aafba4c599 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:56.542875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae076624-8bd0-4f9d-ae6d-87aafba4c599 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:56.556074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae076624-8bd0-4f9d-ae6d-87aafba4c599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-ae076624-8bd0-4f9d-ae6d-87aafba4c599 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:56.565342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:56.566267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 19:39:56.567101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:56.569469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 217/431] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:56.575264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae076624-8bd0-4f9d-ae6d-87aafba4c599 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:56.594062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:56.647042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f43c33-f1e8-4a26-af07-b67975ece8f3 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:56.680523 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:56.684830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae076624-8bd0-4f9d-ae6d-87aafba4c599 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:56.687900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1201fe7-b89f-47c4-b6cd-2f8f08e2da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:56.689574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1201fe7-b89f-47c4-b6cd-2f8f08e2da3a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:56.690070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1201fe7-b89f-47c4-b6cd-2f8f08e2da3a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:56.690541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1201fe7-b89f-47c4-b6cd-2f8f08e2da3a None admin] PATH_INFO: `/v3/projects/574afa1590854c65a807df4589e2c460` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:56.691928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1201fe7-b89f-47c4-b6cd-2f8f08e2da3a None admin] RBAC: Authorizing `identity:get_project(project_id=574afa1590854c65a807df4589e2c460)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:56.693780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1201fe7-b89f-47c4-b6cd-2f8f08e2da3a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:56.696073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 215/432] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:56 2022] GET /identity/v3/projects/574afa1590854c65a807df4589e2c460 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:56.701640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1201fe7-b89f-47c4-b6cd-2f8f08e2da3a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:56.724558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc89232-4e40-4d20-9dc2-34a44a378c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:56.725237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc89232-4e40-4d20-9dc2-34a44a378c65 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:56.725499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc89232-4e40-4d20-9dc2-34a44a378c65 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:56.725814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc89232-4e40-4d20-9dc2-34a44a378c65 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:56.726680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc89232-4e40-4d20-9dc2-34a44a378c65 None admin] RBAC: Adding query filter params (name=member) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:56.726983 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc89232-4e40-4d20-9dc2-34a44a378c65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:56.728470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc89232-4e40-4d20-9dc2-34a44a378c65 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:56.738102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 218/433] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:39:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:56.743575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc89232-4e40-4d20-9dc2-34a44a378c65 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:56.826141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1201fe7-b89f-47c4-b6cd-2f8f08e2da3a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:56.840158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:56.870069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3289b0-e555-4032-bdbf-c807104864f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:56.870912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3289b0-e555-4032-bdbf-c807104864f5 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:56.871188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3289b0-e555-4032-bdbf-c807104864f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:56.871454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3289b0-e555-4032-bdbf-c807104864f5 None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:56.872735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3289b0-e555-4032-bdbf-c807104864f5 None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:56.874215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3289b0-e555-4032-bdbf-c807104864f5 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:56.879287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc89232-4e40-4d20-9dc2-34a44a378c65 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:56.881689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 216/434] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:56 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:56.884012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3289b0-e555-4032-bdbf-c807104864f5 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:56.929210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b1c62-70ad-42ad-88c0-29dea953b33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:56.930592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b1c62-70ad-42ad-88c0-29dea953b33d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:56.931422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b1c62-70ad-42ad-88c0-29dea953b33d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:56.931890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b1c62-70ad-42ad-88c0-29dea953b33d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:56.959741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:57.005158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3289b0-e555-4032-bdbf-c807104864f5 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:57.015023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b1c62-70ad-42ad-88c0-29dea953b33d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:57.016505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b1c62-70ad-42ad-88c0-29dea953b33d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:57.042970 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fe9a7-6ddc-4a49-9026-68e1cf54a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:57.043620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fe9a7-6ddc-4a49-9026-68e1cf54a78d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.043903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fe9a7-6ddc-4a49-9026-68e1cf54a78d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.044179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fe9a7-6ddc-4a49-9026-68e1cf54a78d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.045009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fe9a7-6ddc-4a49-9026-68e1cf54a78d None admin] RBAC: Adding query filter params (group.id=8e1642a69c37473e8e4d86777f84633a, role.id=2e1db0cf27684f389b9320673721684d, scope.project.id=574afa1590854c65a807df4589e2c460) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:57.045328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fe9a7-6ddc-4a49-9026-68e1cf54a78d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:57.046789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fe9a7-6ddc-4a49-9026-68e1cf54a78d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:57.054201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 217/435] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 1 19:39:56 2022] GET /identity/v3/role_assignments?group.id=8e1642a69c37473e8e4d86777f84633a&scope.project.id=574afa1590854c65a807df4589e2c460&role.id=2e1db0cf27684f389b9320673721684d => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:57.055143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8409d-0c9c-432c-9274-31a03ccb8118 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.055431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8409d-0c9c-432c-9274-31a03ccb8118 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.055708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8409d-0c9c-432c-9274-31a03ccb8118 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.057634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 218/436] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:57.062682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81020c-a8f6-4c35-bf77-a3014c3cdf49 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.063084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81020c-a8f6-4c35-bf77-a3014c3cdf49 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.063545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81020c-a8f6-4c35-bf77-a3014c3cdf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.087898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-2d81020c-a8f6-4c35-bf77-a3014c3cdf49 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:57.090154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2d81020c-a8f6-4c35-bf77-a3014c3cdf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:57.109652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:57.144225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 219/437] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:57.149535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b1c62-70ad-42ad-88c0-29dea953b33d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:57.202081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 219/438] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:39:57.226580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81020c-a8f6-4c35-bf77-a3014c3cdf49 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:57.276592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b1c62-70ad-42ad-88c0-29dea953b33d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:57.317856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398104b5-e126-4e55-bdcf-7cc92879b363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:57.318936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398104b5-e126-4e55-bdcf-7cc92879b363 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.319345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398104b5-e126-4e55-bdcf-7cc92879b363 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.319535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:57.319972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398104b5-e126-4e55-bdcf-7cc92879b363 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.321102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398104b5-e126-4e55-bdcf-7cc92879b363 None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:57.322507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398104b5-e126-4e55-bdcf-7cc92879b363 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:57.324550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 220/439] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:57 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:57.330452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398104b5-e126-4e55-bdcf-7cc92879b363 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:57.371697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81020c-a8f6-4c35-bf77-a3014c3cdf49 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:57.394962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:57.413061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc34ab-8d66-4287-9f15-3af195341c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:57.413904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc34ab-8d66-4287-9f15-3af195341c11 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.414184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc34ab-8d66-4287-9f15-3af195341c11 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.414454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc34ab-8d66-4287-9f15-3af195341c11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.427986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc34ab-8d66-4287-9f15-3af195341c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-02cc34ab-8d66-4287-9f15-3af195341c11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:57.436348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:57.437237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:39:57.438063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 220/440] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:39:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:39:57.439628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b01c93-d07f-447c-91bd-e182d599387b None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.439699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b01c93-d07f-447c-91bd-e182d599387b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.439953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b01c93-d07f-447c-91bd-e182d599387b None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.441212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 221/441] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:39:57.447070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ad0c6-9d15-4cd2-b410-6c9b9dde5341 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.447361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ad0c6-9d15-4cd2-b410-6c9b9dde5341 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.447640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ad0c6-9d15-4cd2-b410-6c9b9dde5341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.464968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398104b5-e126-4e55-bdcf-7cc92879b363 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:57.470810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f89ad0c6-9d15-4cd2-b410-6c9b9dde5341 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:39:57.471921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f89ad0c6-9d15-4cd2-b410-6c9b9dde5341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:39:57.504215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8147857-09dd-48ed-adc7-f1be2084fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:57.505345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8147857-09dd-48ed-adc7-f1be2084fc3b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.505650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8147857-09dd-48ed-adc7-f1be2084fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.505928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8147857-09dd-48ed-adc7-f1be2084fc3b None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.507147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8147857-09dd-48ed-adc7-f1be2084fc3b None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:57.508629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8147857-09dd-48ed-adc7-f1be2084fc3b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:57.511743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 221/442] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:57 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:57.511743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8147857-09dd-48ed-adc7-f1be2084fc3b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:57.546307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 222/443] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:39:57.546980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ad0c6-9d15-4cd2-b410-6c9b9dde5341 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:57.627991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:57.638577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8147857-09dd-48ed-adc7-f1be2084fc3b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:57.650944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:57.681347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf634d33-de58-42fe-9715-51c399e766d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:57.682102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634d33-de58-42fe-9715-51c399e766d7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.682383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634d33-de58-42fe-9715-51c399e766d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.682657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634d33-de58-42fe-9715-51c399e766d7 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.683409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf634d33-de58-42fe-9715-51c399e766d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:57.683713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf634d33-de58-42fe-9715-51c399e766d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:57.685085 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf634d33-de58-42fe-9715-51c399e766d7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:57.687350 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ad0c6-9d15-4cd2-b410-6c9b9dde5341 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:57.694560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 222/444] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:39:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:57.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf634d33-de58-42fe-9715-51c399e766d7 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:57.723851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffd00c-902c-4364-9a76-0a448e43d012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:57.724414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffd00c-902c-4364-9a76-0a448e43d012 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.724689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffd00c-902c-4364-9a76-0a448e43d012 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.724957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffd00c-902c-4364-9a76-0a448e43d012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.725805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffd00c-902c-4364-9a76-0a448e43d012 None admin] RBAC: Adding query filter params (group.id=8e1642a69c37473e8e4d86777f84633a, role.id=2e1db0cf27684f389b9320673721684d, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:57.726109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffd00c-902c-4364-9a76-0a448e43d012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:57.727546 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fffd00c-902c-4364-9a76-0a448e43d012 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:57.733205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 223/445] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 1 19:39:57 2022] GET /identity/v3/role_assignments?group.id=8e1642a69c37473e8e4d86777f84633a&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=2e1db0cf27684f389b9320673721684d => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:57.733996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffd00c-902c-4364-9a76-0a448e43d012 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:57.808760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf634d33-de58-42fe-9715-51c399e766d7 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:57.847714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffd00c-902c-4364-9a76-0a448e43d012 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:57.848127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909d2a3-d056-4e27-8801-d976165cc055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:57.848942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909d2a3-d056-4e27-8801-d976165cc055 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.849356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909d2a3-d056-4e27-8801-d976165cc055 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.849660 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909d2a3-d056-4e27-8801-d976165cc055 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.863236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909d2a3-d056-4e27-8801-d976165cc055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:39:57.868151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-8909d2a3-d056-4e27-8801-d976165cc055 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:39:57.871147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:39:57.872050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:39:57.872871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 223/446] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:39:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:39:57.879199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909d2a3-d056-4e27-8801-d976165cc055 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:57.895971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35092418-a1e0-4dc2-a70d-1161b6e01089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:57.896615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35092418-a1e0-4dc2-a70d-1161b6e01089 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:57.896893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35092418-a1e0-4dc2-a70d-1161b6e01089 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:57.897166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35092418-a1e0-4dc2-a70d-1161b6e01089 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:57.915478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:57.993156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35092418-a1e0-4dc2-a70d-1161b6e01089 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:57.995013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35092418-a1e0-4dc2-a70d-1161b6e01089 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:58.006287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909d2a3-d056-4e27-8801-d976165cc055 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:58.061410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38841c15-f1b9-461f-ad4d-b8e09a14bc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:58.062113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38841c15-f1b9-461f-ad4d-b8e09a14bc3e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:58.062401 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38841c15-f1b9-461f-ad4d-b8e09a14bc3e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:58.062666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38841c15-f1b9-461f-ad4d-b8e09a14bc3e None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:58.063440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38841c15-f1b9-461f-ad4d-b8e09a14bc3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:58.063744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38841c15-f1b9-461f-ad4d-b8e09a14bc3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:58.065254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38841c15-f1b9-461f-ad4d-b8e09a14bc3e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:58.073918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 224/447] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:39:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:58.079779 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38841c15-f1b9-461f-ad4d-b8e09a14bc3e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:58.126888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 224/448] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:58.132422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35092418-a1e0-4dc2-a70d-1161b6e01089 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:58.150168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:58.182738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:58.190260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38841c15-f1b9-461f-ad4d-b8e09a14bc3e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:58.223060 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e279de7-80e8-4ae6-b815-d9fa217c86b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:58.223708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e279de7-80e8-4ae6-b815-d9fa217c86b8 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:58.223987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e279de7-80e8-4ae6-b815-d9fa217c86b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:58.224261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e279de7-80e8-4ae6-b815-d9fa217c86b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:58.231798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35092418-a1e0-4dc2-a70d-1161b6e01089 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:58.265257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52575790-3550-4257-85d2-405a1a1e7408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:58.266069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52575790-3550-4257-85d2-405a1a1e7408 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:58.266347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52575790-3550-4257-85d2-405a1a1e7408 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:58.267678 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52575790-3550-4257-85d2-405a1a1e7408 None admin] PATH_INFO: `/v3/projects/3d0bca94154a400bb0d7c93638e9b20f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:58.268874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52575790-3550-4257-85d2-405a1a1e7408 None admin] RBAC: Authorizing `identity:get_project(project_id=3d0bca94154a400bb0d7c93638e9b20f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:58.270337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52575790-3550-4257-85d2-405a1a1e7408 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:58.272001 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 225/449] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:58 2022] GET /identity/v3/projects/3d0bca94154a400bb0d7c93638e9b20f => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:58.277540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52575790-3550-4257-85d2-405a1a1e7408 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:58.295445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e279de7-80e8-4ae6-b815-d9fa217c86b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:58.296922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e279de7-80e8-4ae6-b815-d9fa217c86b8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:58.363870 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:58.380706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52575790-3550-4257-85d2-405a1a1e7408 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:58.396745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:58.405680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 225/450] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:39:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:39:58.411125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e279de7-80e8-4ae6-b815-d9fa217c86b8 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:58.416036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6992d6-e872-42ba-87c0-7fe52f3a1029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:58.416925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6992d6-e872-42ba-87c0-7fe52f3a1029 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:58.417208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6992d6-e872-42ba-87c0-7fe52f3a1029 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:58.417483 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6992d6-e872-42ba-87c0-7fe52f3a1029 None admin] PATH_INFO: `/v3/groups/8e1642a69c37473e8e4d86777f84633a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:58.418724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6992d6-e872-42ba-87c0-7fe52f3a1029 None admin] RBAC: Authorizing `identity:get_group(group_id=8e1642a69c37473e8e4d86777f84633a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:58.420199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6992d6-e872-42ba-87c0-7fe52f3a1029 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:58.421866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 226/451] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:58 2022] GET /identity/v3/groups/8e1642a69c37473e8e4d86777f84633a => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:58.427620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6992d6-e872-42ba-87c0-7fe52f3a1029 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:58.509927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e279de7-80e8-4ae6-b815-d9fa217c86b8 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:58.527866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6992d6-e872-42ba-87c0-7fe52f3a1029 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:58.542863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb4eb5-8339-498d-a19b-4bef928d2039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:58.543722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb4eb5-8339-498d-a19b-4bef928d2039 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:58.544006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb4eb5-8339-498d-a19b-4bef928d2039 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:58.544272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb4eb5-8339-498d-a19b-4bef928d2039 None admin] PATH_INFO: `/v3/projects/54a2d5d499f64a6a8437120d4d4f1f46` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:58.545438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb4eb5-8339-498d-a19b-4bef928d2039 None admin] RBAC: Authorizing `identity:get_project(project_id=54a2d5d499f64a6a8437120d4d4f1f46)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:58.546951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb4eb5-8339-498d-a19b-4bef928d2039 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:58.549171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 226/452] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:39:58 2022] GET /identity/v3/projects/54a2d5d499f64a6a8437120d4d4f1f46 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:58.554510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb4eb5-8339-498d-a19b-4bef928d2039 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:58.564285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d98845-0790-4085-8897-56ffe5f89fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:58.564941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d98845-0790-4085-8897-56ffe5f89fd8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:58.565216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d98845-0790-4085-8897-56ffe5f89fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:58.565482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d98845-0790-4085-8897-56ffe5f89fd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:58.566410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d98845-0790-4085-8897-56ffe5f89fd8 None admin] RBAC: Adding query filter params (group.id=8e1642a69c37473e8e4d86777f84633a, role.id=b127031d393048d7ae9a35015113e82a, scope.project.id=3d0bca94154a400bb0d7c93638e9b20f) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:58.566715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d98845-0790-4085-8897-56ffe5f89fd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:58.568149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d98845-0790-4085-8897-56ffe5f89fd8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:58.573846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 227/453] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 1 19:39:58 2022] GET /identity/v3/role_assignments?group.id=8e1642a69c37473e8e4d86777f84633a&scope.project.id=3d0bca94154a400bb0d7c93638e9b20f&role.id=b127031d393048d7ae9a35015113e82a => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:39:58.618184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:58.661382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb4eb5-8339-498d-a19b-4bef928d2039 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:58.696501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeab34-fad3-4eaf-a818-2a37ebe52be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:58.697393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeab34-fad3-4eaf-a818-2a37ebe52be0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:58.697713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeab34-fad3-4eaf-a818-2a37ebe52be0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:58.697996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeab34-fad3-4eaf-a818-2a37ebe52be0 None admin] PATH_INFO: `/v3/groups/3fb9745e730b470aadf5ea697e5748e5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:58.699229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfeab34-fad3-4eaf-a818-2a37ebe52be0 None admin] RBAC: Authorizing `identity:get_group(group_id=3fb9745e730b470aadf5ea697e5748e5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:58.700712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfeab34-fad3-4eaf-a818-2a37ebe52be0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:58.702424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 227/454] 10.222.0.66 () {56 vars in 1180 bytes} [Fri Jul 1 19:39:58 2022] GET /identity/v3/groups/3fb9745e730b470aadf5ea697e5748e5 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:39:58.709254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d98845-0790-4085-8897-56ffe5f89fd8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:39:58.729256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:39:58.815513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d98845-0790-4085-8897-56ffe5f89fd8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:39:58.850396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fa9b3-0d7b-4057-8323-8159159ef6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:39:58.851043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa9b3-0d7b-4057-8323-8159159ef6ff None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:39:58.851323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa9b3-0d7b-4057-8323-8159159ef6ff None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:39:58.851597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa9b3-0d7b-4057-8323-8159159ef6ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:39:58.852422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fa9b3-0d7b-4057-8323-8159159ef6ff None admin] RBAC: Adding query filter params (group.id=3fb9745e730b470aadf5ea697e5748e5, role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=54a2d5d499f64a6a8437120d4d4f1f46) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:39:58.852730 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fa9b3-0d7b-4057-8323-8159159ef6ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:39:58.854210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fa9b3-0d7b-4057-8323-8159159ef6ff None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:39:58.860129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 228/455] 10.222.0.66 () {56 vars in 1376 bytes} [Fri Jul 1 19:39:58 2022] GET /identity/v3/role_assignments?group.id=3fb9745e730b470aadf5ea697e5748e5&scope.project.id=54a2d5d499f64a6a8437120d4d4f1f46&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:01.259148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd20fdc-1411-4433-a9bf-fe01355f2dd2 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:01.259663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd20fdc-1411-4433-a9bf-fe01355f2dd2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:01.259663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd20fdc-1411-4433-a9bf-fe01355f2dd2 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:01.261306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 228/456] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:01.266172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694342e4-d03e-42ab-a64b-babcaf4be4dc None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:01.266624 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694342e4-d03e-42ab-a64b-babcaf4be4dc None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:01.267419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694342e4-d03e-42ab-a64b-babcaf4be4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:01.290556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-694342e4-d03e-42ab-a64b-babcaf4be4dc None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:01.291767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-694342e4-d03e-42ab-a64b-babcaf4be4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:01.376696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 229/457] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:01.396518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e47c3-7aa1-40a4-a8d4-d725957fbcb4 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:01.396807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e47c3-7aa1-40a4-a8d4-d725957fbcb4 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:01.397084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e47c3-7aa1-40a4-a8d4-d725957fbcb4 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:01.398471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 229/458] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:01.405478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1e2fe-cc71-43b9-8892-bef70305e9bf None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:01.405790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1e2fe-cc71-43b9-8892-bef70305e9bf None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:01.406063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1e2fe-cc71-43b9-8892-bef70305e9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:01.412234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e47c3-7aa1-40a4-a8d4-d725957fbcb4 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:01.433021 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c7e1e2fe-cc71-43b9-8892-bef70305e9bf None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:01.435798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c7e1e2fe-cc71-43b9-8892-bef70305e9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:01.441990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:01.530718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 230/459] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:01.568841 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1e2fe-cc71-43b9-8892-bef70305e9bf None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:01.611753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e47c3-7aa1-40a4-a8d4-d725957fbcb4 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:01.659403 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:01.666722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af1c9d-9b6c-4b3b-a905-ba6ef81981e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:01.667692 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af1c9d-9b6c-4b3b-a905-ba6ef81981e8 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:01.668005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af1c9d-9b6c-4b3b-a905-ba6ef81981e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:01.668376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af1c9d-9b6c-4b3b-a905-ba6ef81981e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:01.680702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af1c9d-9b6c-4b3b-a905-ba6ef81981e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:01.682309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af1c9d-9b6c-4b3b-a905-ba6ef81981e8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-36af1c9d-9b6c-4b3b-a905-ba6ef81981e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:01.703231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:01.704755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:01.705574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:01.705574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:01.705574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:01.705574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:01.710794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 230/460] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:01.712072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd87fd2-d227-41c8-87f0-618d6a935706 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:01.712407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd87fd2-d227-41c8-87f0-618d6a935706 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:01.712621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd87fd2-d227-41c8-87f0-618d6a935706 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:01.714270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 231/461] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:01.715114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd87fd2-d227-41c8-87f0-618d6a935706 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:01.721951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1e2fe-cc71-43b9-8892-bef70305e9bf None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:01.767623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b101f4-c27f-4990-909f-7f26d50dad5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:01.768980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b101f4-c27f-4990-909f-7f26d50dad5b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:01.769746 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b101f4-c27f-4990-909f-7f26d50dad5b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:01.770221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b101f4-c27f-4990-909f-7f26d50dad5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:01.782840 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:01.785243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b101f4-c27f-4990-909f-7f26d50dad5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:01.787919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b101f4-c27f-4990-909f-7f26d50dad5b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-78b101f4-c27f-4990-909f-7f26d50dad5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:01.812439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:01.813453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:01.814313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:01.814313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:01.814313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:01.814313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:01.814313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 231/462] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:01.819829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be774ff-27af-44b1-af30-9c5b85870736 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:01.819829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be774ff-27af-44b1-af30-9c5b85870736 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:01.819829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be774ff-27af-44b1-af30-9c5b85870736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:01.841669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-6be774ff-27af-44b1-af30-9c5b85870736 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:01.843424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6be774ff-27af-44b1-af30-9c5b85870736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:01.843736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd87fd2-d227-41c8-87f0-618d6a935706 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:01.882255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b94b2f-e99a-4749-9d09-d692691acd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:01.882916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b94b2f-e99a-4749-9d09-d692691acd3e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:01.883324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b94b2f-e99a-4749-9d09-d692691acd3e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:01.883562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b94b2f-e99a-4749-9d09-d692691acd3e None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:01.884341 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b94b2f-e99a-4749-9d09-d692691acd3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:01.884871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b94b2f-e99a-4749-9d09-d692691acd3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:01.886149 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b94b2f-e99a-4749-9d09-d692691acd3e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:01.904106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 232/463] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:01.905070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b94b2f-e99a-4749-9d09-d692691acd3e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:01.935113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 232/464] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:01.936969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be774ff-27af-44b1-af30-9c5b85870736 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:01.975564 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:02.041232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b94b2f-e99a-4749-9d09-d692691acd3e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:02.073194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be774ff-27af-44b1-af30-9c5b85870736 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:02.075526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04387d09-4776-420f-a02d-11b2b82b2e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:02.076197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04387d09-4776-420f-a02d-11b2b82b2e41 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.077200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04387d09-4776-420f-a02d-11b2b82b2e41 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.077200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04387d09-4776-420f-a02d-11b2b82b2e41 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:02.077468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04387d09-4776-420f-a02d-11b2b82b2e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:02.077788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04387d09-4776-420f-a02d-11b2b82b2e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:02.079840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04387d09-4776-420f-a02d-11b2b82b2e41 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:02.096581 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 233/465] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:02.096581 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04387d09-4776-420f-a02d-11b2b82b2e41 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:02.109662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:02.125541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1bdd1-d74c-4912-a0fd-320ccbff9810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:02.127874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1bdd1-d74c-4912-a0fd-320ccbff9810 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.128377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1bdd1-d74c-4912-a0fd-320ccbff9810 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.129010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1bdd1-d74c-4912-a0fd-320ccbff9810 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:02.130281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1bdd1-d74c-4912-a0fd-320ccbff9810 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:02.132145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1bdd1-d74c-4912-a0fd-320ccbff9810 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:02.159273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 233/466] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:40:01 2022] POST /identity/v3/users => generated 246 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:02.160388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1bdd1-d74c-4912-a0fd-320ccbff9810 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:02.235969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04387d09-4776-420f-a02d-11b2b82b2e41 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:02.237978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:02.277198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6787b-25b0-4d82-8d67-13f74a5f9b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:02.278062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6787b-25b0-4d82-8d67-13f74a5f9b8c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.278365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6787b-25b0-4d82-8d67-13f74a5f9b8c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.278624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6787b-25b0-4d82-8d67-13f74a5f9b8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:02.293368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6787b-25b0-4d82-8d67-13f74a5f9b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:02.295036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6787b-25b0-4d82-8d67-13f74a5f9b8c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-5bf6787b-25b0-4d82-8d67-13f74a5f9b8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:02.324110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:02.325041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:02.325905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:02.325905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:02.325905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:02.325905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:02.325905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 234/467] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:02.327343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2718be-49a9-4a10-829a-47c972695fb9 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.327636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2718be-49a9-4a10-829a-47c972695fb9 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.327972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2718be-49a9-4a10-829a-47c972695fb9 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:02.333666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 235/468] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:02.334518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2718be-49a9-4a10-829a-47c972695fb9 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:02.350186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1bdd1-d74c-4912-a0fd-320ccbff9810 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:02.394620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8aa296-45fa-4cd9-9478-39163a287c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:02.395310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aa296-45fa-4cd9-9478-39163a287c8f None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.395586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aa296-45fa-4cd9-9478-39163a287c8f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.395866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aa296-45fa-4cd9-9478-39163a287c8f None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:02.396893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8aa296-45fa-4cd9-9478-39163a287c8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:02.398370 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8aa296-45fa-4cd9-9478-39163a287c8f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:02.424029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 234/469] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:40:02 2022] POST /identity/v3/users => generated 248 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:02.425374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9d96e-fa7b-4ae3-9121-3652ea00bc99 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.425687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9d96e-fa7b-4ae3-9121-3652ea00bc99 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.425963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9d96e-fa7b-4ae3-9121-3652ea00bc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:02.445410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f0b9d96e-fa7b-4ae3-9121-3652ea00bc99 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:02.446915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f0b9d96e-fa7b-4ae3-9121-3652ea00bc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:02.467520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2718be-49a9-4a10-829a-47c972695fb9 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:02.494017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:02.539274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418e3e2-f4da-4a13-96fb-ad7e95b42e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:02.539967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418e3e2-f4da-4a13-96fb-ad7e95b42e6c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.540353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418e3e2-f4da-4a13-96fb-ad7e95b42e6c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.540628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418e3e2-f4da-4a13-96fb-ad7e95b42e6c None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:02.541375 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418e3e2-f4da-4a13-96fb-ad7e95b42e6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:02.541713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418e3e2-f4da-4a13-96fb-ad7e95b42e6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:02.547034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418e3e2-f4da-4a13-96fb-ad7e95b42e6c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:02.568899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 236/470] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:02.574300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418e3e2-f4da-4a13-96fb-ad7e95b42e6c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:02.579769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 235/471] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:02.592861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:02.618358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9d96e-fa7b-4ae3-9121-3652ea00bc99 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:02.722390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418e3e2-f4da-4a13-96fb-ad7e95b42e6c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:02.765509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9d96e-fa7b-4ae3-9121-3652ea00bc99 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:02.765720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092a137-aba1-4aa4-a06f-08498557df03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:02.766618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a137-aba1-4aa4-a06f-08498557df03 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.766618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a137-aba1-4aa4-a06f-08498557df03 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.766916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092a137-aba1-4aa4-a06f-08498557df03 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:02.767704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092a137-aba1-4aa4-a06f-08498557df03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:02.769277 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092a137-aba1-4aa4-a06f-08498557df03 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:02.797432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 237/472] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:40:02 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:40:02.810603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cc5cf-abe7-458b-9d95-f544b506ef18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:02.811427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cc5cf-abe7-458b-9d95-f544b506ef18 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.811704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cc5cf-abe7-458b-9d95-f544b506ef18 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.811981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cc5cf-abe7-458b-9d95-f544b506ef18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:02.832039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cc5cf-abe7-458b-9d95-f544b506ef18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:02.833607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cc5cf-abe7-458b-9d95-f544b506ef18 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-112cc5cf-abe7-458b-9d95-f544b506ef18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:02.856238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:02.857179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:02.858048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:02.858048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:02.858048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:02.858048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:02.858048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 236/473] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:02.864457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092a137-aba1-4aa4-a06f-08498557df03 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:02.955696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:02.983901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2c885-c01c-4b00-bf2b-c0c0147fdac2 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.984200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2c885-c01c-4b00-bf2b-c0c0147fdac2 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.984448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2c885-c01c-4b00-bf2b-c0c0147fdac2 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:02.986592 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 237/474] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:02.993207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae0eb9-1608-4cd9-ba79-600360c37a14 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:02.993491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae0eb9-1608-4cd9-ba79-600360c37a14 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:02.993775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae0eb9-1608-4cd9-ba79-600360c37a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:03.003835 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092a137-aba1-4aa4-a06f-08498557df03 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:03.011706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-fcae0eb9-1608-4cd9-ba79-600360c37a14 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:03.012913 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fcae0eb9-1608-4cd9-ba79-600360c37a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:03.026162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:03.041537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40944d98-48db-404b-8913-d43566e13410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:03.042240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40944d98-48db-404b-8913-d43566e13410 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:03.042521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40944d98-48db-404b-8913-d43566e13410 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:03.042797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40944d98-48db-404b-8913-d43566e13410 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:03.043655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40944d98-48db-404b-8913-d43566e13410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:03.044066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40944d98-48db-404b-8913-d43566e13410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:03.045757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40944d98-48db-404b-8913-d43566e13410 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:03.059757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 238/475] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:03.065558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40944d98-48db-404b-8913-d43566e13410 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:03.101972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 238/476] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:03.139448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae0eb9-1608-4cd9-ba79-600360c37a14 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:03.186340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40944d98-48db-404b-8913-d43566e13410 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:03.226262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:03.232440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff26997-0fdd-48c8-afca-7de251436d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:03.233329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff26997-0fdd-48c8-afca-7de251436d9c None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:03.233793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff26997-0fdd-48c8-afca-7de251436d9c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:03.234501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff26997-0fdd-48c8-afca-7de251436d9c None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:03.237001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff26997-0fdd-48c8-afca-7de251436d9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:03.239352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff26997-0fdd-48c8-afca-7de251436d9c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:03.265170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 239/477] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:40:03 2022] POST /identity/v3/users => generated 249 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:40:03.327455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae0eb9-1608-4cd9-ba79-600360c37a14 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:03.371411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9b17f-20b2-4182-b25d-f5b269ae2c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:03.372144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b17f-20b2-4182-b25d-f5b269ae2c9d None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:03.372433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b17f-20b2-4182-b25d-f5b269ae2c9d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:03.372708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b17f-20b2-4182-b25d-f5b269ae2c9d None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:03.373485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9b17f-20b2-4182-b25d-f5b269ae2c9d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:03.374932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9b17f-20b2-4182-b25d-f5b269ae2c9d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:03.383423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.sql.core [None req-01e9b17f-20b2-4182-b25d-f5b269ae2c9d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 19:40:03.383423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 01 19:40:03.383423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [parameters: {'id': '913556dc3f8f42fbad2c5f68422d860a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 01 19:40:03.383423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86779) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-01e9b17f-20b2-4182-b25d-f5b269ae2c9d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application conn.query(q) Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result.read() Jul 01 19:40:03.401336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return ref.to_dict() Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 19:40:03.402288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application next(self.gen) Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application yield resource Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application next(self.gen) Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise self.value Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application session.commit() Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 19:40:03.403164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.session.flush() Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application compat.raise_( Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 19:40:03.404014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 19:40:03.404859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application conn.query(q) Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result.read() Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 19:40:03.406668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [parameters: {'id': '913556dc3f8f42fbad2c5f68422d860a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:03.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 01 19:40:03.411911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:03.421080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 239/478] 10.222.0.66 () {60 vars in 1131 bytes} [Fri Jul 1 19:40:03 2022] POST /identity/v3/roles => generated 144 bytes in 264 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:03.421235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff26997-0fdd-48c8-afca-7de251436d9c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:03.543577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff26997-0fdd-48c8-afca-7de251436d9c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:03.583805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27653765-a6f8-48e9-b741-b45728885f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:03.584600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27653765-a6f8-48e9-b741-b45728885f4f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:03.584881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27653765-a6f8-48e9-b741-b45728885f4f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:03.585159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27653765-a6f8-48e9-b741-b45728885f4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:03.596569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27653765-a6f8-48e9-b741-b45728885f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-27653765-a6f8-48e9-b741-b45728885f4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:03.605072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:03.605972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:03.606750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 240/479] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:40:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:03.612837 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9b17f-20b2-4182-b25d-f5b269ae2c9d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:03.637658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:03.734812 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9b17f-20b2-4182-b25d-f5b269ae2c9d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:03.772198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cca25-627a-4d5b-ba55-5b8e3570ab90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:03.773300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cca25-627a-4d5b-ba55-5b8e3570ab90 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:03.773889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cca25-627a-4d5b-ba55-5b8e3570ab90 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:03.774394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cca25-627a-4d5b-ba55-5b8e3570ab90 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:03.775397 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cca25-627a-4d5b-ba55-5b8e3570ab90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:03.775910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cca25-627a-4d5b-ba55-5b8e3570ab90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:03.777520 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cca25-627a-4d5b-ba55-5b8e3570ab90 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:03.792134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 240/480] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:05.020245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935b65-b47e-4c77-b38d-d69115d2967d None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.020756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935b65-b47e-4c77-b38d-d69115d2967d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.020831 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935b65-b47e-4c77-b38d-d69115d2967d None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.022438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 241/481] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:05.027828 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6d9f7-ac44-4374-9a5d-7024bc5d5f6f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.028632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6d9f7-ac44-4374-9a5d-7024bc5d5f6f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.029125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6d9f7-ac44-4374-9a5d-7024bc5d5f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.050263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7cc6d9f7-ac44-4374-9a5d-7024bc5d5f6f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:05.051785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7cc6d9f7-ac44-4374-9a5d-7024bc5d5f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:05.147510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 241/482] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:05.196105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5935b65-b47e-4c77-b38d-d69115d2967d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:05.218592 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef50cf-4e91-415d-a9cc-0dcd008eb210 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.218868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef50cf-4e91-415d-a9cc-0dcd008eb210 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.219152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef50cf-4e91-415d-a9cc-0dcd008eb210 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.221439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 242/483] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:05.226454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b84bd-42fb-4f78-bec5-e23957d1a213 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.226674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b84bd-42fb-4f78-bec5-e23957d1a213 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.226903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b84bd-42fb-4f78-bec5-e23957d1a213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.233629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:05.251059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-771b84bd-42fb-4f78-bec5-e23957d1a213 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:05.252957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-771b84bd-42fb-4f78-bec5-e23957d1a213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:05.267749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:05.374682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5935b65-b47e-4c77-b38d-d69115d2967d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:05.382231 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 243/484] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:40:05.437346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b84bd-42fb-4f78-bec5-e23957d1a213 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:05.444613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c6b73-4796-43cc-a38d-2d1facce8b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:05.445448 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c6b73-4796-43cc-a38d-2d1facce8b55 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.445771 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c6b73-4796-43cc-a38d-2d1facce8b55 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.446045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c6b73-4796-43cc-a38d-2d1facce8b55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.461773 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c6b73-4796-43cc-a38d-2d1facce8b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-574c6b73-4796-43cc-a38d-2d1facce8b55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:05.472858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:05.473871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:05.474661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:05.476226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 242/485] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:05.478043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9cac5-f6af-46e9-b758-db524334cf7a None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.478565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9cac5-f6af-46e9-b758-db524334cf7a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.479037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9cac5-f6af-46e9-b758-db524334cf7a None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.481352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 243/486] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:05.482690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9cac5-f6af-46e9-b758-db524334cf7a None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:05.580071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b84bd-42fb-4f78-bec5-e23957d1a213 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:05.618669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9cac5-f6af-46e9-b758-db524334cf7a None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:05.620539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:05.627779 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:05.632579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c191af-1e12-4e4a-8ba3-215f27d254ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:05.633974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c191af-1e12-4e4a-8ba3-215f27d254ce None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.635169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c191af-1e12-4e4a-8ba3-215f27d254ce None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.636241 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c191af-1e12-4e4a-8ba3-215f27d254ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.651367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c191af-1e12-4e4a-8ba3-215f27d254ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:05.656541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b54316-2da7-45d0-b0db-05c2166ee80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:05.657818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b54316-2da7-45d0-b0db-05c2166ee80a None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.658102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b54316-2da7-45d0-b0db-05c2166ee80a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.658376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b54316-2da7-45d0-b0db-05c2166ee80a None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.659125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b54316-2da7-45d0-b0db-05c2166ee80a None admin] RBAC: Adding query filter params (name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:05.659431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b54316-2da7-45d0-b0db-05c2166ee80a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:05.661121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b54316-2da7-45d0-b0db-05c2166ee80a None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-a8c191af-1e12-4e4a-8ba3-215f27d254ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:05.663701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:05.664589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:05.665357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:05.674202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 244/488] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:05.675052 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 244/488] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:05.675941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b54316-2da7-45d0-b0db-05c2166ee80a None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:05.676343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46187d05-9c43-4d6f-9b33-9919dde93071 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.676823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46187d05-9c43-4d6f-9b33-9919dde93071 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.677317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46187d05-9c43-4d6f-9b33-9919dde93071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.696188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-46187d05-9c43-4d6f-9b33-9919dde93071 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:05.697384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-46187d05-9c43-4d6f-9b33-9919dde93071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:05.770440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 245/489] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:05.778310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46187d05-9c43-4d6f-9b33-9919dde93071 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:05.805733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b54316-2da7-45d0-b0db-05c2166ee80a None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:05.850719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e2510-ac75-4194-bfd5-fa0af1ac383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:05.851548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2510-ac75-4194-bfd5-fa0af1ac383f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.851828 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2510-ac75-4194-bfd5-fa0af1ac383f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.852105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2510-ac75-4194-bfd5-fa0af1ac383f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.864276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e2510-ac75-4194-bfd5-fa0af1ac383f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:05.866758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e2510-ac75-4194-bfd5-fa0af1ac383f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-f16e2510-ac75-4194-bfd5-fa0af1ac383f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:05.890858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:05.892601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:05.893431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:05.893431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:05.893431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:05.893431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:05.899753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 245/490] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:05.900653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e2510-ac75-4194-bfd5-fa0af1ac383f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:05.908597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46187d05-9c43-4d6f-9b33-9919dde93071 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:05.934058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:05.940323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:05.953360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa47942-191f-4df9-b5fb-7df8624e132e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:05.954902 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa47942-191f-4df9-b5fb-7df8624e132e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:05.955390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa47942-191f-4df9-b5fb-7df8624e132e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:05.955891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa47942-191f-4df9-b5fb-7df8624e132e None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:05.956882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa47942-191f-4df9-b5fb-7df8624e132e None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:05.957388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa47942-191f-4df9-b5fb-7df8624e132e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:05.958994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa47942-191f-4df9-b5fb-7df8624e132e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:05.968124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 246/491] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:05.969090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa47942-191f-4df9-b5fb-7df8624e132e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.026852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e2510-ac75-4194-bfd5-fa0af1ac383f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.062885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900cff5-3f64-442e-b089-7e527066b77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.064450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900cff5-3f64-442e-b089-7e527066b77a None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.064735 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900cff5-3f64-442e-b089-7e527066b77a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.065008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900cff5-3f64-442e-b089-7e527066b77a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.075428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900cff5-3f64-442e-b089-7e527066b77a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:06.078263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa47942-191f-4df9-b5fb-7df8624e132e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6900cff5-3f64-442e-b089-7e527066b77a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:06.083193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:06.084072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:06.085282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 246/492] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:06.086171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767ff4f-dff3-495b-a584-f36dc5af76fa None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.086240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767ff4f-dff3-495b-a584-f36dc5af76fa None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.086490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767ff4f-dff3-495b-a584-f36dc5af76fa None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.087910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 247/493] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:06.089705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767ff4f-dff3-495b-a584-f36dc5af76fa None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.118956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2e7d7-d5d5-4ac5-9c7c-eddb0368a103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.119611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2e7d7-d5d5-4ac5-9c7c-eddb0368a103 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.119890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2e7d7-d5d5-4ac5-9c7c-eddb0368a103 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.120162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2e7d7-d5d5-4ac5-9c7c-eddb0368a103 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.121008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2e7d7-d5d5-4ac5-9c7c-eddb0368a103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:06.121450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2e7d7-d5d5-4ac5-9c7c-eddb0368a103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:06.122858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2e7d7-d5d5-4ac5-9c7c-eddb0368a103 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:06.140554 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 247/494] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:06.141795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528143b-77a3-4ad7-8ba1-ba1f601028ce None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.142086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528143b-77a3-4ad7-8ba1-ba1f601028ce None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.142470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528143b-77a3-4ad7-8ba1-ba1f601028ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.159352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-3528143b-77a3-4ad7-8ba1-ba1f601028ce None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:06.160529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3528143b-77a3-4ad7-8ba1-ba1f601028ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:06.180868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:06.201857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767ff4f-dff3-495b-a584-f36dc5af76fa None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.208322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:06.237878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 248/495] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:06.238912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528143b-77a3-4ad7-8ba1-ba1f601028ce None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.247763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f910f-3c87-4b13-8a6b-4e30abfc251c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.249200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f910f-3c87-4b13-8a6b-4e30abfc251c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.249671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f910f-3c87-4b13-8a6b-4e30abfc251c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.250119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f910f-3c87-4b13-8a6b-4e30abfc251c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.264958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f910f-3c87-4b13-8a6b-4e30abfc251c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:06.266542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f910f-3c87-4b13-8a6b-4e30abfc251c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-f78f910f-3c87-4b13-8a6b-4e30abfc251c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:06.285620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:06.286920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:06.292349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:06.292349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:06.292349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:06.292349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:06.292349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 248/496] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:06.292349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f910f-3c87-4b13-8a6b-4e30abfc251c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.354109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528143b-77a3-4ad7-8ba1-ba1f601028ce None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.390444 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad895f-74a6-49c7-9041-d246a8e27a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.391109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad895f-74a6-49c7-9041-d246a8e27a87 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.392580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad895f-74a6-49c7-9041-d246a8e27a87 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.392806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad895f-74a6-49c7-9041-d246a8e27a87 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.393548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad895f-74a6-49c7-9041-d246a8e27a87 None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:06.393877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad895f-74a6-49c7-9041-d246a8e27a87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:06.395256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad895f-74a6-49c7-9041-d246a8e27a87 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:06.396922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f910f-3c87-4b13-8a6b-4e30abfc251c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.403537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 249/497] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:06.408868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad895f-74a6-49c7-9041-d246a8e27a87 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.427201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:06.432282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af27d1-4d7b-4d75-8c12-529bd818d727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.433062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af27d1-4d7b-4d75-8c12-529bd818d727 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.433346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af27d1-4d7b-4d75-8c12-529bd818d727 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.433637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af27d1-4d7b-4d75-8c12-529bd818d727 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.441892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af27d1-4d7b-4d75-8c12-529bd818d727 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:06.443342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af27d1-4d7b-4d75-8c12-529bd818d727 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:06.445522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-18af27d1-4d7b-4d75-8c12-529bd818d727 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:06.460365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:06.461480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:06.462423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 249/498] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:06.463690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af27d1-4d7b-4d75-8c12-529bd818d727 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.511374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad895f-74a6-49c7-9041-d246a8e27a87 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.544802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c944832-1bad-49c4-a268-0c0b327e03f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.545595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c944832-1bad-49c4-a268-0c0b327e03f2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.545899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c944832-1bad-49c4-a268-0c0b327e03f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.546174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c944832-1bad-49c4-a268-0c0b327e03f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.559081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c944832-1bad-49c4-a268-0c0b327e03f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:06.565923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af27d1-4d7b-4d75-8c12-529bd818d727 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-3c944832-1bad-49c4-a268-0c0b327e03f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:06.567430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:06.568384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:06.569162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 250/499] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:06.569832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c944832-1bad-49c4-a268-0c0b327e03f2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.602698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aaf2b7-2e5f-45a7-9d6a-e9e1d1fc04bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.603356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaf2b7-2e5f-45a7-9d6a-e9e1d1fc04bd None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.603659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaf2b7-2e5f-45a7-9d6a-e9e1d1fc04bd None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.603934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aaf2b7-2e5f-45a7-9d6a-e9e1d1fc04bd None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.604655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaf2b7-2e5f-45a7-9d6a-e9e1d1fc04bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:06.604960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaf2b7-2e5f-45a7-9d6a-e9e1d1fc04bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:06.606372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aaf2b7-2e5f-45a7-9d6a-e9e1d1fc04bd None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:06.617745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 250/500] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:06.619158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e5236-c0a3-4ae4-aba9-98088146581c None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.619450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e5236-c0a3-4ae4-aba9-98088146581c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.619725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e5236-c0a3-4ae4-aba9-98088146581c None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.621124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 251/501] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:06.622715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e5236-c0a3-4ae4-aba9-98088146581c None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.663611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:06.671873 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c944832-1bad-49c4-a268-0c0b327e03f2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.679541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:06.706731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec77a06-8b94-4ef9-80ab-ef4c41fd92e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.707510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec77a06-8b94-4ef9-80ab-ef4c41fd92e3 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.707789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec77a06-8b94-4ef9-80ab-ef4c41fd92e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.708062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec77a06-8b94-4ef9-80ab-ef4c41fd92e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.719274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec77a06-8b94-4ef9-80ab-ef4c41fd92e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:06.720866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec77a06-8b94-4ef9-80ab-ef4c41fd92e3 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:06.729320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e5236-c0a3-4ae4-aba9-98088146581c None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-3ec77a06-8b94-4ef9-80ab-ef4c41fd92e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:06.738499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:06.739545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:06.740463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:06.740463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:06.740463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:06.740463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:06.740463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 251/502] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:06.740986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec77a06-8b94-4ef9-80ab-ef4c41fd92e3 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.770025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8a625-4c67-4441-96a9-27f94a60d51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.770692 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8a625-4c67-4441-96a9-27f94a60d51f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.770971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8a625-4c67-4441-96a9-27f94a60d51f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.771242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8a625-4c67-4441-96a9-27f94a60d51f None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.772080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8a625-4c67-4441-96a9-27f94a60d51f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:06.772381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8a625-4c67-4441-96a9-27f94a60d51f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:06.773855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8a625-4c67-4441-96a9-27f94a60d51f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:06.810103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 252/503] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:06.810943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8a625-4c67-4441-96a9-27f94a60d51f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.856189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec77a06-8b94-4ef9-80ab-ef4c41fd92e3 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.892878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f44da1-ab11-4cfa-a0a3-72cc72fec1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.893538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f44da1-ab11-4cfa-a0a3-72cc72fec1ea None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.893844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f44da1-ab11-4cfa-a0a3-72cc72fec1ea None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.894113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f44da1-ab11-4cfa-a0a3-72cc72fec1ea None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.894849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f44da1-ab11-4cfa-a0a3-72cc72fec1ea None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:06.895149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f44da1-ab11-4cfa-a0a3-72cc72fec1ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:06.896516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f44da1-ab11-4cfa-a0a3-72cc72fec1ea None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:06.904521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 252/504] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:06.905698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a4685-56e9-4dae-a31c-a9b470338777 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.905979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a4685-56e9-4dae-a31c-a9b470338777 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.906251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a4685-56e9-4dae-a31c-a9b470338777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.910186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8a625-4c67-4441-96a9-27f94a60d51f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:06.923173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:06.925360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d36a4685-56e9-4dae-a31c-a9b470338777 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:06.926594 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d36a4685-56e9-4dae-a31c-a9b470338777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:06.929292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:06.947577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510027b6-3e2a-4875-8f78-def826a70167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:06.948364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510027b6-3e2a-4875-8f78-def826a70167 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:06.948642 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510027b6-3e2a-4875-8f78-def826a70167 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:06.948911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510027b6-3e2a-4875-8f78-def826a70167 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:06.956541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510027b6-3e2a-4875-8f78-def826a70167 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:06.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510027b6-3e2a-4875-8f78-def826a70167 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-510027b6-3e2a-4875-8f78-def826a70167 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:06.972611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:06.973515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:06.974437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 253/505] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:06.974982 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510027b6-3e2a-4875-8f78-def826a70167 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:06.996150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 253/506] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:07.002853 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a4685-56e9-4dae-a31c-a9b470338777 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:07.076164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510027b6-3e2a-4875-8f78-def826a70167 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.100447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a4685-56e9-4dae-a31c-a9b470338777 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.119909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40216f-08eb-4b0d-8d54-bb41933002eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.120568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40216f-08eb-4b0d-8d54-bb41933002eb None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.120850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40216f-08eb-4b0d-8d54-bb41933002eb None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.121122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40216f-08eb-4b0d-8d54-bb41933002eb None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.121886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40216f-08eb-4b0d-8d54-bb41933002eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:07.122188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40216f-08eb-4b0d-8d54-bb41933002eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:07.123559 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40216f-08eb-4b0d-8d54-bb41933002eb None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:07.134911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258aaa0d-e6f2-4ca3-9414-3c7ade25d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.135101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 254/507] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:07.135556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aaa0d-e6f2-4ca3-9414-3c7ade25d57f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.135838 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aaa0d-e6f2-4ca3-9414-3c7ade25d57f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.136008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40216f-08eb-4b0d-8d54-bb41933002eb None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:07.136300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aaa0d-e6f2-4ca3-9414-3c7ade25d57f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.148912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:07.174666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:07.207658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258aaa0d-e6f2-4ca3-9414-3c7ade25d57f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:07.209109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258aaa0d-e6f2-4ca3-9414-3c7ade25d57f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:07.243524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40216f-08eb-4b0d-8d54-bb41933002eb None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.276554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f2f48-fda7-45bf-8cba-3060db9a4fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.277599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f2f48-fda7-45bf-8cba-3060db9a4fb4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.277905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f2f48-fda7-45bf-8cba-3060db9a4fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.278174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f2f48-fda7-45bf-8cba-3060db9a4fb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.289047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f2f48-fda7-45bf-8cba-3060db9a4fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:07.290526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f2f48-fda7-45bf-8cba-3060db9a4fb4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:07.307747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 254/508] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-c25f2f48-fda7-45bf-8cba-3060db9a4fb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:07.307877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:07.308637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:07.309460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:07.309460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:07.309460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:07.309460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:07.310144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258aaa0d-e6f2-4ca3-9414-3c7ade25d57f None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:07.310216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 255/509] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:07.310216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f2f48-fda7-45bf-8cba-3060db9a4fb4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:07.353858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:07.391661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:07.410482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f2f48-fda7-45bf-8cba-3060db9a4fb4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.410680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258aaa0d-e6f2-4ca3-9414-3c7ade25d57f None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.443774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6104d8-34d5-4211-b01f-9ff119f755ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.444481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6104d8-34d5-4211-b01f-9ff119f755ff None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.444766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6104d8-34d5-4211-b01f-9ff119f755ff None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.444854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4ee28-3618-4c0b-8d16-2e9d77bcf1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.445187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6104d8-34d5-4211-b01f-9ff119f755ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.445471 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4ee28-3618-4c0b-8d16-2e9d77bcf1ef None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.445785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4ee28-3618-4c0b-8d16-2e9d77bcf1ef None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.446060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4ee28-3618-4c0b-8d16-2e9d77bcf1ef None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.446903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4ee28-3618-4c0b-8d16-2e9d77bcf1ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:07.447213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4ee28-3618-4c0b-8d16-2e9d77bcf1ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:07.448687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4ee28-3618-4c0b-8d16-2e9d77bcf1ef None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:07.457410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6104d8-34d5-4211-b01f-9ff119f755ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:07.458944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6104d8-34d5-4211-b01f-9ff119f755ff None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-ea6104d8-34d5-4211-b01f-9ff119f755ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:07.476826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:07.477908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:07.478750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:07.478750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:07.478750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:07.478750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:07.478750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 256/510] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:07.479006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6104d8-34d5-4211-b01f-9ff119f755ff None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:07.489538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 255/511] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 1 19:40:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:07.490391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4ee28-3618-4c0b-8d16-2e9d77bcf1ef None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:07.587239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6104d8-34d5-4211-b01f-9ff119f755ff None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.589096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:07.593271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4ee28-3618-4c0b-8d16-2e9d77bcf1ef None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.620657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd562e6e-81ca-4520-9e5a-c6219d56f645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.621450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd562e6e-81ca-4520-9e5a-c6219d56f645 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.621777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd562e6e-81ca-4520-9e5a-c6219d56f645 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.622051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd562e6e-81ca-4520-9e5a-c6219d56f645 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.623941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:07.628529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96f9d0-51ff-460a-b4fb-7f414f9d9397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.629313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96f9d0-51ff-460a-b4fb-7f414f9d9397 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.629619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96f9d0-51ff-460a-b4fb-7f414f9d9397 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.629894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96f9d0-51ff-460a-b4fb-7f414f9d9397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.630948 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd562e6e-81ca-4520-9e5a-c6219d56f645 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:07.632628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd562e6e-81ca-4520-9e5a-c6219d56f645 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:07.639783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96f9d0-51ff-460a-b4fb-7f414f9d9397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:07.641834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96f9d0-51ff-460a-b4fb-7f414f9d9397 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-bd562e6e-81ca-4520-9e5a-c6219d56f645 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:07.647132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:07.648128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:07.648998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 257/512] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:07.649554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd562e6e-81ca-4520-9e5a-c6219d56f645 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-ef96f9d0-51ff-460a-b4fb-7f414f9d9397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:07.660334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:07.661552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:07.662454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:07.662454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:07.662454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:07.662454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:07.662454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 256/513] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:07.662716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96f9d0-51ff-460a-b4fb-7f414f9d9397 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:07.753910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd562e6e-81ca-4520-9e5a-c6219d56f645 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.762878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96f9d0-51ff-460a-b4fb-7f414f9d9397 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.797047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f4c0b-6e7a-4e92-a7f0-689ac3f95d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.797874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f4c0b-6e7a-4e92-a7f0-689ac3f95d83 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.798025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c81e6-77f1-4789-b43a-a374e1502bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.798168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f4c0b-6e7a-4e92-a7f0-689ac3f95d83 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.798419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f4c0b-6e7a-4e92-a7f0-689ac3f95d83 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.798581 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c81e6-77f1-4789-b43a-a374e1502bad None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.798866 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c81e6-77f1-4789-b43a-a374e1502bad None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.799142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c81e6-77f1-4789-b43a-a374e1502bad None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.799231 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f4c0b-6e7a-4e92-a7f0-689ac3f95d83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:07.799447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f4c0b-6e7a-4e92-a7f0-689ac3f95d83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:07.799880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c81e6-77f1-4789-b43a-a374e1502bad None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:07.800188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c81e6-77f1-4789-b43a-a374e1502bad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:07.800868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2f4c0b-6e7a-4e92-a7f0-689ac3f95d83 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:07.801602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c81e6-77f1-4789-b43a-a374e1502bad None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:07.813331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 257/514] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:07.818081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 258/515] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:07.818204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f4c0b-6e7a-4e92-a7f0-689ac3f95d83 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:07.818304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c81e6-77f1-4789-b43a-a374e1502bad None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:07.829956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:07.871682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:07.917027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c81e6-77f1-4789-b43a-a374e1502bad None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.917831 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f4c0b-6e7a-4e92-a7f0-689ac3f95d83 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:07.950450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c433986-98ec-4c5c-8163-ef3de14a7b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.951110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c433986-98ec-4c5c-8163-ef3de14a7b36 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.951389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c433986-98ec-4c5c-8163-ef3de14a7b36 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.951668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c433986-98ec-4c5c-8163-ef3de14a7b36 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.951831 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e822d8-c0b1-447e-b1a8-d7801204e364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:07.952471 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e822d8-c0b1-447e-b1a8-d7801204e364 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:07.952560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c433986-98ec-4c5c-8163-ef3de14a7b36 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:07.952752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e822d8-c0b1-447e-b1a8-d7801204e364 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:07.952840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c433986-98ec-4c5c-8163-ef3de14a7b36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:07.953029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e822d8-c0b1-447e-b1a8-d7801204e364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:07.954999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c433986-98ec-4c5c-8163-ef3de14a7b36 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:07.972028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 259/516] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 1 19:40:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:07.972824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c433986-98ec-4c5c-8163-ef3de14a7b36 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:08.021710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e822d8-c0b1-447e-b1a8-d7801204e364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:08.023120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e822d8-c0b1-447e-b1a8-d7801204e364 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:08.034599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:08.073081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c433986-98ec-4c5c-8163-ef3de14a7b36 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:08.086026 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:08.107361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba85c5a-d248-4e32-89df-2aadffbfaeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:08.108020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba85c5a-d248-4e32-89df-2aadffbfaeb5 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:08.108305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba85c5a-d248-4e32-89df-2aadffbfaeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:08.108580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba85c5a-d248-4e32-89df-2aadffbfaeb5 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:08.109300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba85c5a-d248-4e32-89df-2aadffbfaeb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:08.109632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba85c5a-d248-4e32-89df-2aadffbfaeb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:08.111629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba85c5a-d248-4e32-89df-2aadffbfaeb5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:08.122865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 260/517] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:08.128356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 258/518] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:08.128837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba85c5a-d248-4e32-89df-2aadffbfaeb5 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:08.128924 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e822d8-c0b1-447e-b1a8-d7801204e364 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:08.223583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e822d8-c0b1-447e-b1a8-d7801204e364 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:08.228446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba85c5a-d248-4e32-89df-2aadffbfaeb5 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:08.233765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:08.260148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b4735-49cd-494f-ad3a-7c37392704ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:08.260930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b4735-49cd-494f-ad3a-7c37392704ae None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:08.261206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b4735-49cd-494f-ad3a-7c37392704ae None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:08.261479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b4735-49cd-494f-ad3a-7c37392704ae None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:08.262914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49cb1f-f726-4d67-80bc-8c0a11e01c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:08.263575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49cb1f-f726-4d67-80bc-8c0a11e01c69 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:08.263856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49cb1f-f726-4d67-80bc-8c0a11e01c69 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:08.264138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49cb1f-f726-4d67-80bc-8c0a11e01c69 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:08.264841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49cb1f-f726-4d67-80bc-8c0a11e01c69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:08.266599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49cb1f-f726-4d67-80bc-8c0a11e01c69 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:08.268783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b4735-49cd-494f-ad3a-7c37392704ae None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:08.270256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b4735-49cd-494f-ad3a-7c37392704ae None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:08.286886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 261/519] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:40:08 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-2d1b4735-49cd-494f-ad3a-7c37392704ae None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:08.287017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:08.287874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:08.288740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:08.288740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:08.288740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:08.288740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:08.288740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:08.288740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:08.288740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:08.288740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:08.288740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 19:40:08.288740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:08.289235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49cb1f-f726-4d67-80bc-8c0a11e01c69 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:08.289302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 259/520] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:08.289302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b4735-49cd-494f-ad3a-7c37392704ae None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:08.325084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:08.411177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49cb1f-f726-4d67-80bc-8c0a11e01c69 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:08.415724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b4735-49cd-494f-ad3a-7c37392704ae None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:08.449803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef5bb6-66a4-4586-8bc2-cdd7386e9214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:08.450757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef5bb6-66a4-4586-8bc2-cdd7386e9214 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:08.451105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef5bb6-66a4-4586-8bc2-cdd7386e9214 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:08.451380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef5bb6-66a4-4586-8bc2-cdd7386e9214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:08.459059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f008f0-3f92-41fa-bbf6-0ff2a8004d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:08.459942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f008f0-3f92-41fa-bbf6-0ff2a8004d61 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:08.460293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f008f0-3f92-41fa-bbf6-0ff2a8004d61 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:08.461057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f008f0-3f92-41fa-bbf6-0ff2a8004d61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:08.470230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f008f0-3f92-41fa-bbf6-0ff2a8004d61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:08.472094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f008f0-3f92-41fa-bbf6-0ff2a8004d61 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-47f008f0-3f92-41fa-bbf6-0ff2a8004d61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:08.489868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:08.490894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:08.491944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:08.491944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:08.491944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:08.491944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:08.491944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:08.491944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:08.491944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:08.491944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:08.493291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 260/521] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:08.494258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f008f0-3f92-41fa-bbf6-0ff2a8004d61 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:08.526551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef5bb6-66a4-4586-8bc2-cdd7386e9214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:08.528134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef5bb6-66a4-4586-8bc2-cdd7386e9214 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:08.529420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:08.561654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:08.609460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f008f0-3f92-41fa-bbf6-0ff2a8004d61 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:08.642495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 262/522] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:08.643658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef5bb6-66a4-4586-8bc2-cdd7386e9214 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:08.645927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4d6b8-59b0-461b-af98-7fea2d6c3fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:08.646726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4d6b8-59b0-461b-af98-7fea2d6c3fff None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:08.647010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4d6b8-59b0-461b-af98-7fea2d6c3fff None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:08.647287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4d6b8-59b0-461b-af98-7fea2d6c3fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:08.659054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4d6b8-59b0-461b-af98-7fea2d6c3fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:08.660574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4d6b8-59b0-461b-af98-7fea2d6c3fff None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-91f4d6b8-59b0-461b-af98-7fea2d6c3fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:08.679232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:08.680115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:08.680941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:08.680941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:08.680941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:08.680941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:08.680941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 261/523] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:08.681335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4d6b8-59b0-461b-af98-7fea2d6c3fff None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:08.755658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef5bb6-66a4-4586-8bc2-cdd7386e9214 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:08.769632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:08.787568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:08.790104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4d6b8-59b0-461b-af98-7fea2d6c3fff None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:08.795829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8da05-4a71-4281-83d2-f1334c557548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:08.796730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8da05-4a71-4281-83d2-f1334c557548 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:08.797186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8da05-4a71-4281-83d2-f1334c557548 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:08.797912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8da05-4a71-4281-83d2-f1334c557548 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:08.799328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8da05-4a71-4281-83d2-f1334c557548 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:08.799640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8da05-4a71-4281-83d2-f1334c557548 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:08.801473 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8da05-4a71-4281-83d2-f1334c557548 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:08.825677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 263/524] 10.222.0.66 () {56 vars in 1139 bytes} [Fri Jul 1 19:40:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:08.825677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8da05-4a71-4281-83d2-f1334c557548 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:08.825957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ec18-c6da-4a28-a226-dc28d8951266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:08.825957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ec18-c6da-4a28-a226-dc28d8951266 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:08.825957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ec18-c6da-4a28-a226-dc28d8951266 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:08.826207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ec18-c6da-4a28-a226-dc28d8951266 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:08.826896 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746ec18-c6da-4a28-a226-dc28d8951266 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:08.827302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746ec18-c6da-4a28-a226-dc28d8951266 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:08.830357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746ec18-c6da-4a28-a226-dc28d8951266 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:08.855390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 262/525] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:08.855390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ec18-c6da-4a28-a226-dc28d8951266 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:08.934802 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8da05-4a71-4281-83d2-f1334c557548 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:08.966366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ec18-c6da-4a28-a226-dc28d8951266 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:08.971644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5150d14-15ac-45dd-b38f-fee109d20100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:08.972440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5150d14-15ac-45dd-b38f-fee109d20100 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:08.972751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5150d14-15ac-45dd-b38f-fee109d20100 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:08.973024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5150d14-15ac-45dd-b38f-fee109d20100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:08.983632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5150d14-15ac-45dd-b38f-fee109d20100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:08.985896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5150d14-15ac-45dd-b38f-fee109d20100 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:09.002682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da43233-2205-4154-b465-6a3e6fc5343c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:09.003345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da43233-2205-4154-b465-6a3e6fc5343c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:09.003628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da43233-2205-4154-b465-6a3e6fc5343c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:09.003903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da43233-2205-4154-b465-6a3e6fc5343c None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:09.004623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da43233-2205-4154-b465-6a3e6fc5343c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:09.004929 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da43233-2205-4154-b465-6a3e6fc5343c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-a5150d14-15ac-45dd-b38f-fee109d20100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:09.005021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:09.005922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:09.006763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:09.006763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:09.006763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:09.006763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:09.006763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 264/526] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:09.007453 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5150d14-15ac-45dd-b38f-fee109d20100 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:09.010839 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da43233-2205-4154-b465-6a3e6fc5343c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:09.027989 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 263/527] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:09.028904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da43233-2205-4154-b465-6a3e6fc5343c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:09.032093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:09.051501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:09.119826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5150d14-15ac-45dd-b38f-fee109d20100 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:09.141460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da43233-2205-4154-b465-6a3e6fc5343c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:09.160967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb5b9a-debd-4ada-aa74-ce618c4ebec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:09.161643 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb5b9a-debd-4ada-aa74-ce618c4ebec7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:09.161929 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb5b9a-debd-4ada-aa74-ce618c4ebec7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:09.162200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb5b9a-debd-4ada-aa74-ce618c4ebec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:09.177396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb6ee4-c9f2-4aea-ac75-fcee471a617a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:09.178073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb6ee4-c9f2-4aea-ac75-fcee471a617a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:09.178357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb6ee4-c9f2-4aea-ac75-fcee471a617a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:09.178635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb6ee4-c9f2-4aea-ac75-fcee471a617a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:09.179463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb6ee4-c9f2-4aea-ac75-fcee471a617a None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=ad06f31859f9413f9db6039ac2abf982) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:09.179782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb6ee4-c9f2-4aea-ac75-fcee471a617a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:09.181224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb6ee4-c9f2-4aea-ac75-fcee471a617a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:09.186676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 264/528] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:09 2022] GET /identity/v3/role_assignments?user.id=ad06f31859f9413f9db6039ac2abf982&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:09.187450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb6ee4-c9f2-4aea-ac75-fcee471a617a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:09.257538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb5b9a-debd-4ada-aa74-ce618c4ebec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:09.259183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb5b9a-debd-4ada-aa74-ce618c4ebec7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:09.286905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:09.312028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:09.335067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb6ee4-c9f2-4aea-ac75-fcee471a617a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:09.381511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fd42e-1043-4a27-82d7-4922cec497fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:09.382938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fd42e-1043-4a27-82d7-4922cec497fa None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:09.383426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fd42e-1043-4a27-82d7-4922cec497fa None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:09.383985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494fd42e-1043-4a27-82d7-4922cec497fa None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:09.385190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fd42e-1043-4a27-82d7-4922cec497fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:09.385951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fd42e-1043-4a27-82d7-4922cec497fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:09.387968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494fd42e-1043-4a27-82d7-4922cec497fa None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:09.399601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 265/529] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:09.400671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb5b9a-debd-4ada-aa74-ce618c4ebec7 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:09.403030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 265/530] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:09.404006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fd42e-1043-4a27-82d7-4922cec497fa None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:09.516507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb5b9a-debd-4ada-aa74-ce618c4ebec7 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:09.532762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494fd42e-1043-4a27-82d7-4922cec497fa None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:09.554327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d1c7a-84c5-4abe-a92e-604783a36e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:09.555407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d1c7a-84c5-4abe-a92e-604783a36e3e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:09.555791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d1c7a-84c5-4abe-a92e-604783a36e3e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:09.556185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d1c7a-84c5-4abe-a92e-604783a36e3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:09.564152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d1c7a-84c5-4abe-a92e-604783a36e3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:09.565894 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d1c7a-84c5-4abe-a92e-604783a36e3e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:09.567977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:09.569137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:09.578263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfc269-7400-4ed6-bce1-f50dce8bf00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:09.580135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfc269-7400-4ed6-bce1-f50dce8bf00d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:09.580568 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfc269-7400-4ed6-bce1-f50dce8bf00d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:09.581036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfc269-7400-4ed6-bce1-f50dce8bf00d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-525d1c7a-84c5-4abe-a92e-604783a36e3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:09.585674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:09.586724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:09.587658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:09.587658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:09.587658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:09.587658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:09.587658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:09.587658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:09.587658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:09.587658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:09.593248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 266/531] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:09.595653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d1c7a-84c5-4abe-a92e-604783a36e3e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:09.597126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfc269-7400-4ed6-bce1-f50dce8bf00d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:09.598854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfc269-7400-4ed6-bce1-f50dce8bf00d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-63bfc269-7400-4ed6-bce1-f50dce8bf00d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:09.617798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:09.618685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:09.619524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:09.619524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:09.619524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:09.619524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:09.621726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 266/532] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:09.622725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfc269-7400-4ed6-bce1-f50dce8bf00d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:09.712203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d1c7a-84c5-4abe-a92e-604783a36e3e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:09.749407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c3ad9-7777-43a2-8f65-19e0e8670dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:09.750267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c3ad9-7777-43a2-8f65-19e0e8670dd0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:09.751169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c3ad9-7777-43a2-8f65-19e0e8670dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:09.751407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfc269-7400-4ed6-bce1-f50dce8bf00d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:09.751536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c3ad9-7777-43a2-8f65-19e0e8670dd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:09.758537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c3ad9-7777-43a2-8f65-19e0e8670dd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:09.759988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c3ad9-7777-43a2-8f65-19e0e8670dd0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-b12c3ad9-7777-43a2-8f65-19e0e8670dd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:09.775678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:09.776570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:09.777400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:09.777400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:09.777400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:09.777400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:09.777400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:09.777400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:09.777400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:09.777400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:09.781502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 267/533] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:09.782613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c3ad9-7777-43a2-8f65-19e0e8670dd0 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:09.798950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7180635-1158-41dd-b49b-e6d8e7ad5ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:09.800364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7180635-1158-41dd-b49b-e6d8e7ad5ed6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:09.801075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7180635-1158-41dd-b49b-e6d8e7ad5ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:09.801533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7180635-1158-41dd-b49b-e6d8e7ad5ed6 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:09.802790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7180635-1158-41dd-b49b-e6d8e7ad5ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:09.803726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7180635-1158-41dd-b49b-e6d8e7ad5ed6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:09.805409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7180635-1158-41dd-b49b-e6d8e7ad5ed6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:09.822554 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 267/534] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:09.823624 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7180635-1158-41dd-b49b-e6d8e7ad5ed6 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:09.850245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:09.874144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:09.905509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c3ad9-7777-43a2-8f65-19e0e8670dd0 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:09.945263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0df7d-96ac-4792-9875-74a9c1afd0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:09.946015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0df7d-96ac-4792-9875-74a9c1afd0fb None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:09.946367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0df7d-96ac-4792-9875-74a9c1afd0fb None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:09.946714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0df7d-96ac-4792-9875-74a9c1afd0fb None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:09.947557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0df7d-96ac-4792-9875-74a9c1afd0fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:09.947932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0df7d-96ac-4792-9875-74a9c1afd0fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:09.948467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7180635-1158-41dd-b49b-e6d8e7ad5ed6 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:09.949482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c0df7d-96ac-4792-9875-74a9c1afd0fb None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:09.963986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 268/535] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:09.969096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0df7d-96ac-4792-9875-74a9c1afd0fb None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:09.989542 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7793d-de2a-4064-b8b1-0f83efe39db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:09.990555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7793d-de2a-4064-b8b1-0f83efe39db0 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:09.990826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7793d-de2a-4064-b8b1-0f83efe39db0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:09.991178 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7793d-de2a-4064-b8b1-0f83efe39db0 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:09.992103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7793d-de2a-4064-b8b1-0f83efe39db0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:09.992594 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7793d-de2a-4064-b8b1-0f83efe39db0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:09.994282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7793d-de2a-4064-b8b1-0f83efe39db0 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:10.008040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 268/536] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:10.013758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7793d-de2a-4064-b8b1-0f83efe39db0 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:10.087730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c0df7d-96ac-4792-9875-74a9c1afd0fb None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:10.106366 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:10.124232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:10.125237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d328278-2417-42b8-93fc-75acb2e37f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:10.127103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d328278-2417-42b8-93fc-75acb2e37f7b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:10.127803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d328278-2417-42b8-93fc-75acb2e37f7b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:10.128537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d328278-2417-42b8-93fc-75acb2e37f7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:10.129680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d328278-2417-42b8-93fc-75acb2e37f7b None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=c022224b03a54ab489d95502c63a25dd) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:10.130067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7793d-de2a-4064-b8b1-0f83efe39db0 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:10.130453 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d328278-2417-42b8-93fc-75acb2e37f7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:10.132648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d328278-2417-42b8-93fc-75acb2e37f7b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:10.139494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 269/537] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:09 2022] GET /identity/v3/role_assignments?user.id=c022224b03a54ab489d95502c63a25dd&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:10.140530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d328278-2417-42b8-93fc-75acb2e37f7b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:10.196484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4cb8-47b5-4b45-bbba-75eeba0399fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:10.197335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4cb8-47b5-4b45-bbba-75eeba0399fe None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:10.197646 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4cb8-47b5-4b45-bbba-75eeba0399fe None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:10.197926 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4cb8-47b5-4b45-bbba-75eeba0399fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:10.210673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda4cb8-47b5-4b45-bbba-75eeba0399fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:10.212082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda4cb8-47b5-4b45-bbba-75eeba0399fe None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-ceda4cb8-47b5-4b45-bbba-75eeba0399fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:10.240499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:10.241393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:10.242275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:10.242275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:10.242275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:10.242275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:10.242275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:10.242275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:10.242275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:10.242275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:10.246477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 269/538] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:10.248369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4cb8-47b5-4b45-bbba-75eeba0399fe None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:10.256480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d328278-2417-42b8-93fc-75acb2e37f7b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:10.294863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f22b42-c41f-4fa1-b141-cc97c6b37f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:10.295939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f22b42-c41f-4fa1-b141-cc97c6b37f67 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:10.296437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f22b42-c41f-4fa1-b141-cc97c6b37f67 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:10.296925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f22b42-c41f-4fa1-b141-cc97c6b37f67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:10.298623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f22b42-c41f-4fa1-b141-cc97c6b37f67 None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=a4b0e6ea57af454cbc8f5bd180bc052e) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:10.299158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f22b42-c41f-4fa1-b141-cc97c6b37f67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:10.300878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f22b42-c41f-4fa1-b141-cc97c6b37f67 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:10.312212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 270/539] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:10 2022] GET /identity/v3/role_assignments?user.id=a4b0e6ea57af454cbc8f5bd180bc052e&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:10.427231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379d962-6db3-462a-b368-a94a861172fd None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:10.427563 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379d962-6db3-462a-b368-a94a861172fd None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:10.427825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379d962-6db3-462a-b368-a94a861172fd None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:10.429395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 271/540] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:10.435539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbd0fb-da5c-48ee-b16d-292f8309f415 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:10.435880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbd0fb-da5c-48ee-b16d-292f8309f415 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:10.436097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbd0fb-da5c-48ee-b16d-292f8309f415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:10.436259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4cb8-47b5-4b45-bbba-75eeba0399fe None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:10.465976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f1fbd0fb-da5c-48ee-b16d-292f8309f415 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:10.467198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f1fbd0fb-da5c-48ee-b16d-292f8309f415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:10.475431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2fbbe-3cb6-4a00-ab7c-eec701ea4158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:10.476220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2fbbe-3cb6-4a00-ab7c-eec701ea4158 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:10.476569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2fbbe-3cb6-4a00-ab7c-eec701ea4158 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:10.476912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2fbbe-3cb6-4a00-ab7c-eec701ea4158 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:10.477822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2fbbe-3cb6-4a00-ab7c-eec701ea4158 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:10.478210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2fbbe-3cb6-4a00-ab7c-eec701ea4158 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:10.479773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2fbbe-3cb6-4a00-ab7c-eec701ea4158 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:10.516657 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 270/541] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:10.523085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2fbbe-3cb6-4a00-ab7c-eec701ea4158 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:10.533838 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:10.565279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 272/542] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:10.574435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:10.601450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbd0fb-da5c-48ee-b16d-292f8309f415 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:10.666410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2fbbe-3cb6-4a00-ab7c-eec701ea4158 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:10.702111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba38fcb-5ec3-4acd-aae8-7eefad237234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:10.703136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba38fcb-5ec3-4acd-aae8-7eefad237234 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:10.704143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba38fcb-5ec3-4acd-aae8-7eefad237234 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:10.704684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba38fcb-5ec3-4acd-aae8-7eefad237234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:10.705871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba38fcb-5ec3-4acd-aae8-7eefad237234 None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=370f1c0f7af04a56bdf8fa452550380a) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:10.706409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba38fcb-5ec3-4acd-aae8-7eefad237234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:10.708074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba38fcb-5ec3-4acd-aae8-7eefad237234 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:10.720559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 271/543] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:10 2022] GET /identity/v3/role_assignments?user.id=370f1c0f7af04a56bdf8fa452550380a&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:10.739781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbd0fb-da5c-48ee-b16d-292f8309f415 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:10.780719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42108e73-5be3-4f49-bc18-1a9cd12443be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:10.781454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42108e73-5be3-4f49-bc18-1a9cd12443be None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:10.782718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42108e73-5be3-4f49-bc18-1a9cd12443be None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:10.783069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42108e73-5be3-4f49-bc18-1a9cd12443be None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:10.796168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42108e73-5be3-4f49-bc18-1a9cd12443be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-42108e73-5be3-4f49-bc18-1a9cd12443be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:10.804711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:10.805691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:10.806701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 273/544] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:10.813702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba38fcb-5ec3-4acd-aae8-7eefad237234 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:10.930282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:10.951265 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba38fcb-5ec3-4acd-aae8-7eefad237234 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:11.005056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdd334-39da-4a56-9850-c4602b14d747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:11.005755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdd334-39da-4a56-9850-c4602b14d747 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:11.006038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdd334-39da-4a56-9850-c4602b14d747 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:11.006310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdd334-39da-4a56-9850-c4602b14d747 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:11.007060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdd334-39da-4a56-9850-c4602b14d747 None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:11.007365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdd334-39da-4a56-9850-c4602b14d747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:11.008753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdd334-39da-4a56-9850-c4602b14d747 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:11.021719 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 272/545] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:11.027153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42108e73-5be3-4f49-bc18-1a9cd12443be None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:11.083731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:11.159425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42108e73-5be3-4f49-bc18-1a9cd12443be None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:11.196648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb99f1f-4497-4a65-8010-d6ab520e4832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:11.197437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb99f1f-4497-4a65-8010-d6ab520e4832 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:11.197734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb99f1f-4497-4a65-8010-d6ab520e4832 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:11.198006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb99f1f-4497-4a65-8010-d6ab520e4832 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:11.209492 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb99f1f-4497-4a65-8010-d6ab520e4832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:11.210973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb99f1f-4497-4a65-8010-d6ab520e4832 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6fb99f1f-4497-4a65-8010-d6ab520e4832 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:11.230010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:11.230881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:11.231890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:11.231890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:11.231890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:11.231890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:11.231890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 274/546] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:11.238251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdd334-39da-4a56-9850-c4602b14d747 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:11.389535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdd334-39da-4a56-9850-c4602b14d747 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:11.436606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68b8ae-dd2d-444c-abf2-5209c4d5b53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:11.437270 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b8ae-dd2d-444c-abf2-5209c4d5b53c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:11.437552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b8ae-dd2d-444c-abf2-5209c4d5b53c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:11.437933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b8ae-dd2d-444c-abf2-5209c4d5b53c None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:11.438665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68b8ae-dd2d-444c-abf2-5209c4d5b53c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:11.438971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68b8ae-dd2d-444c-abf2-5209c4d5b53c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:11.440341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68b8ae-dd2d-444c-abf2-5209c4d5b53c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:11.457987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 273/547] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:11.463466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb99f1f-4497-4a65-8010-d6ab520e4832 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:11.473694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c495955-c87f-4603-bd18-5813d7e16b13 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:11.473977 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c495955-c87f-4603-bd18-5813d7e16b13 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:11.474254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c495955-c87f-4603-bd18-5813d7e16b13 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:11.475859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 274/548] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:11.480842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aea570-cde1-4956-85e9-5d8af1531792 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:11.481129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aea570-cde1-4956-85e9-5d8af1531792 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:11.481406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aea570-cde1-4956-85e9-5d8af1531792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:11.492871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:11.499653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d9aea570-cde1-4956-85e9-5d8af1531792 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:11.501155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d9aea570-cde1-4956-85e9-5d8af1531792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:11.598063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:11.610204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb99f1f-4497-4a65-8010-d6ab520e4832 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:11.626058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 275/549] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:11.646004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3017f-96f9-483b-83fc-9b448fbe0608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:11.647078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3017f-96f9-483b-83fc-9b448fbe0608 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:11.647405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3017f-96f9-483b-83fc-9b448fbe0608 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:11.647668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3017f-96f9-483b-83fc-9b448fbe0608 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:11.655358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3017f-96f9-483b-83fc-9b448fbe0608 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:11.656360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aea570-cde1-4956-85e9-5d8af1531792 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:11.656570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3017f-96f9-483b-83fc-9b448fbe0608 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-5cc3017f-96f9-483b-83fc-9b448fbe0608 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:11.671725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:11.673168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:11.674405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 275/550] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:11.681314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3017f-96f9-483b-83fc-9b448fbe0608 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:11.787636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3017f-96f9-483b-83fc-9b448fbe0608 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:11.821965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dad1f-0eac-416c-a457-e6e653542715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:11.822634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dad1f-0eac-416c-a457-e6e653542715 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:11.822918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dad1f-0eac-416c-a457-e6e653542715 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:11.823191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dad1f-0eac-416c-a457-e6e653542715 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:11.824027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dad1f-0eac-416c-a457-e6e653542715 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:11.824331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dad1f-0eac-416c-a457-e6e653542715 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:11.825781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dad1f-0eac-416c-a457-e6e653542715 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:11.843886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 276/551] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 1 19:40:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:11.850605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dad1f-0eac-416c-a457-e6e653542715 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:11.861889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:11.932638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aea570-cde1-4956-85e9-5d8af1531792 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:11.973202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dad1f-0eac-416c-a457-e6e653542715 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:12.016858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04fd62-3436-4c95-976e-153154fed48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:12.017680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04fd62-3436-4c95-976e-153154fed48b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:12.017956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04fd62-3436-4c95-976e-153154fed48b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:12.018225 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04fd62-3436-4c95-976e-153154fed48b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:12.021887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:12.035628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04fd62-3436-4c95-976e-153154fed48b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:12.041807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6ba9-194c-463e-bf56-970dd7ce572f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:12.042449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6ba9-194c-463e-bf56-970dd7ce572f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:12.042725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6ba9-194c-463e-bf56-970dd7ce572f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:12.042994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b6ba9-194c-463e-bf56-970dd7ce572f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-0c04fd62-3436-4c95-976e-153154fed48b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:12.051405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:12.052380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:12.053274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:12.057352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 276/552] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:12.058487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04fd62-3436-4c95-976e-153154fed48b None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:12.171961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b6ba9-194c-463e-bf56-970dd7ce572f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:12.174041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b6ba9-194c-463e-bf56-970dd7ce572f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:12.184160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:12.255575 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04fd62-3436-4c95-976e-153154fed48b None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:12.316770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aad0f-e720-4830-bbb6-609ff52fe88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:12.317490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aad0f-e720-4830-bbb6-609ff52fe88e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:12.317799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aad0f-e720-4830-bbb6-609ff52fe88e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:12.318074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aad0f-e720-4830-bbb6-609ff52fe88e None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:12.318824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607aad0f-e720-4830-bbb6-609ff52fe88e None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:12.319131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607aad0f-e720-4830-bbb6-609ff52fe88e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:12.320525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607aad0f-e720-4830-bbb6-609ff52fe88e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:12.337343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 277/553] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:12.341085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aad0f-e720-4830-bbb6-609ff52fe88e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:12.358370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 277/554] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:12.364030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6ba9-194c-463e-bf56-970dd7ce572f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:12.454464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:12.539624 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aad0f-e720-4830-bbb6-609ff52fe88e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:12.558522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b6ba9-194c-463e-bf56-970dd7ce572f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:12.594076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:12.599216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495bc6f-a328-4444-b9bd-a9172bb68556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:12.600669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495bc6f-a328-4444-b9bd-a9172bb68556 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:12.601187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495bc6f-a328-4444-b9bd-a9172bb68556 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:12.601739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495bc6f-a328-4444-b9bd-a9172bb68556 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:12.613373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495bc6f-a328-4444-b9bd-a9172bb68556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:12.615553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495bc6f-a328-4444-b9bd-a9172bb68556 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-d495bc6f-a328-4444-b9bd-a9172bb68556 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:12.635395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:12.636489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:12.637626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:12.637626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:12.637626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:12.637626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:12.639477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 278/555] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:12.639876 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a8fff-256f-49b3-a45e-08fdb9323831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:12.640676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a8fff-256f-49b3-a45e-08fdb9323831 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:12.641502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a8fff-256f-49b3-a45e-08fdb9323831 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:12.641810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a8fff-256f-49b3-a45e-08fdb9323831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:12.650057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495bc6f-a328-4444-b9bd-a9172bb68556 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:12.668098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a8fff-256f-49b3-a45e-08fdb9323831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:12.669610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a8fff-256f-49b3-a45e-08fdb9323831 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-468a8fff-256f-49b3-a45e-08fdb9323831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:12.700108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:12.701042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:12.702435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:12.702435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:12.702435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:12.702435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:12.707266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 278/556] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:12.708474 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a8fff-256f-49b3-a45e-08fdb9323831 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:12.806085 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495bc6f-a328-4444-b9bd-a9172bb68556 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:12.853334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbe24f-cf59-45aa-a561-5bd2e3fc16f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:12.853757 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a8fff-256f-49b3-a45e-08fdb9323831 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:12.854052 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe24f-cf59-45aa-a561-5bd2e3fc16f5 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:12.854336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe24f-cf59-45aa-a561-5bd2e3fc16f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:12.854613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe24f-cf59-45aa-a561-5bd2e3fc16f5 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:12.855658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbe24f-cf59-45aa-a561-5bd2e3fc16f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:12.856892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbe24f-cf59-45aa-a561-5bd2e3fc16f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:12.858357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbe24f-cf59-45aa-a561-5bd2e3fc16f5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:12.864684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:12.881988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 279/557] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:12.883080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc785785-d9fe-4be2-a266-fc047a132685 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:12.883365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc785785-d9fe-4be2-a266-fc047a132685 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:12.883639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc785785-d9fe-4be2-a266-fc047a132685 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:12.886307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 280/558] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:12.886307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc785785-d9fe-4be2-a266-fc047a132685 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:12.904654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f17fb9-87bc-474b-a18f-862c6ac8ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:12.905327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f17fb9-87bc-474b-a18f-862c6ac8ac51 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:12.905647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f17fb9-87bc-474b-a18f-862c6ac8ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:12.905906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f17fb9-87bc-474b-a18f-862c6ac8ac51 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:12.907067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f17fb9-87bc-474b-a18f-862c6ac8ac51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:12.907534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f17fb9-87bc-474b-a18f-862c6ac8ac51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:12.909211 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f17fb9-87bc-474b-a18f-862c6ac8ac51 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:12.915749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:12.931076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 279/559] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:12.931076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647f6cb-3149-4d5f-88c2-f46f03c0a2c1 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:12.931076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647f6cb-3149-4d5f-88c2-f46f03c0a2c1 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:12.931076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647f6cb-3149-4d5f-88c2-f46f03c0a2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:12.958040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-4647f6cb-3149-4d5f-88c2-f46f03c0a2c1 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:12.959830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4647f6cb-3149-4d5f-88c2-f46f03c0a2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:13.027163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc785785-d9fe-4be2-a266-fc047a132685 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:13.061665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 280/560] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:13.061665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647f6cb-3149-4d5f-88c2-f46f03c0a2c1 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:13.067691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94934bf-dfda-4c5b-88d4-67c3f1bd8dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:13.068488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94934bf-dfda-4c5b-88d4-67c3f1bd8dd2 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.068814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94934bf-dfda-4c5b-88d4-67c3f1bd8dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.069088 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94934bf-dfda-4c5b-88d4-67c3f1bd8dd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.076424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94934bf-dfda-4c5b-88d4-67c3f1bd8dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:13.078034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94934bf-dfda-4c5b-88d4-67c3f1bd8dd2 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-e94934bf-dfda-4c5b-88d4-67c3f1bd8dd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:13.093620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:13.095109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:13.096025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:13.096025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:13.096025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:13.096025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:13.096025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:13.096025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:13.096025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:13.096025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:13.096025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 281/561] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:13.097101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89e621-fee5-4d62-a88d-6812714fb92c None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.097279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89e621-fee5-4d62-a88d-6812714fb92c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.097548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89e621-fee5-4d62-a88d-6812714fb92c None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.099000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 282/562] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:13.102376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89e621-fee5-4d62-a88d-6812714fb92c None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:13.176172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:13.180523 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647f6cb-3149-4d5f-88c2-f46f03c0a2c1 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:13.194911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:13.217314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a78fce-0c90-4bd6-89e8-e0be6d903c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:13.218155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a78fce-0c90-4bd6-89e8-e0be6d903c16 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.218514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a78fce-0c90-4bd6-89e8-e0be6d903c16 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.218889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a78fce-0c90-4bd6-89e8-e0be6d903c16 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.220352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89e621-fee5-4d62-a88d-6812714fb92c None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:13.226588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a78fce-0c90-4bd6-89e8-e0be6d903c16 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:13.228062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a78fce-0c90-4bd6-89e8-e0be6d903c16 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-27a78fce-0c90-4bd6-89e8-e0be6d903c16 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:13.242584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:13.243536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:13.244462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 281/563] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 1 19:40:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:13.245072 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a78fce-0c90-4bd6-89e8-e0be6d903c16 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:13.259581 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab027765-b825-4631-8d31-34c507db3765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:13.260381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027765-b825-4631-8d31-34c507db3765 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.260662 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027765-b825-4631-8d31-34c507db3765 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.260933 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027765-b825-4631-8d31-34c507db3765 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.271939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab027765-b825-4631-8d31-34c507db3765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-ab027765-b825-4631-8d31-34c507db3765 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:13.279702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:13.280764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:13.281686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 283/564] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:13.282363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f9151-c998-44f5-a827-9bd3266f1405 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.282550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f9151-c998-44f5-a827-9bd3266f1405 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.282822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f9151-c998-44f5-a827-9bd3266f1405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.299158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-878f9151-c998-44f5-a827-9bd3266f1405 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:13.300328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-878f9151-c998-44f5-a827-9bd3266f1405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:13.350891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a78fce-0c90-4bd6-89e8-e0be6d903c16 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:13.365745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 284/565] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:13.367037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13791be-63e4-477a-a73f-e7a4a46023f8 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.367328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13791be-63e4-477a-a73f-e7a4a46023f8 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.367606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13791be-63e4-477a-a73f-e7a4a46023f8 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.369010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 285/566] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:13.373604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13791be-63e4-477a-a73f-e7a4a46023f8 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:13.384587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9e200-90c6-4caf-9941-c936e3648c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:13.385240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9e200-90c6-4caf-9941-c936e3648c51 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.385518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9e200-90c6-4caf-9941-c936e3648c51 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.385819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9e200-90c6-4caf-9941-c936e3648c51 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.386605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9e200-90c6-4caf-9941-c936e3648c51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:13.386909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9e200-90c6-4caf-9941-c936e3648c51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:13.388343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9e200-90c6-4caf-9941-c936e3648c51 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:13.401866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 282/567] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:13.402662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9e200-90c6-4caf-9941-c936e3648c51 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:13.421205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:13.445158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:13.486895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13791be-63e4-477a-a73f-e7a4a46023f8 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:13.504957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9e200-90c6-4caf-9941-c936e3648c51 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:13.526653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be287a7-bb07-42b4-b32e-ccb6c15ecd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:13.527318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be287a7-bb07-42b4-b32e-ccb6c15ecd27 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.527599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be287a7-bb07-42b4-b32e-ccb6c15ecd27 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.527876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be287a7-bb07-42b4-b32e-ccb6c15ecd27 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.528715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be287a7-bb07-42b4-b32e-ccb6c15ecd27 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:13.529017 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be287a7-bb07-42b4-b32e-ccb6c15ecd27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:13.530497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be287a7-bb07-42b4-b32e-ccb6c15ecd27 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:13.539233 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf2a24-b2d8-491b-9990-9d455d89a398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:13.539898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf2a24-b2d8-491b-9990-9d455d89a398 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.540209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf2a24-b2d8-491b-9990-9d455d89a398 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.540481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf2a24-b2d8-491b-9990-9d455d89a398 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.541220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf2a24-b2d8-491b-9990-9d455d89a398 None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:13.541525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf2a24-b2d8-491b-9990-9d455d89a398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:13.542926 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf2a24-b2d8-491b-9990-9d455d89a398 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:13.551812 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 283/568] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:13.553260 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833daf-e486-438f-9c51-977212576a79 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.553560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833daf-e486-438f-9c51-977212576a79 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.553861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833daf-e486-438f-9c51-977212576a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.556774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 286/569] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 1 19:40:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:13.557695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be287a7-bb07-42b4-b32e-ccb6c15ecd27 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:13.570930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-8a833daf-e486-438f-9c51-977212576a79 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:13.572144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8a833daf-e486-438f-9c51-977212576a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:13.642317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 284/570] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:13.643150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a833daf-e486-438f-9c51-977212576a79 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:13.654216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:13.659541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be287a7-bb07-42b4-b32e-ccb6c15ecd27 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:13.687165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:13.697213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df9ead-4b64-4e0a-8542-6e8439903141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:13.698033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df9ead-4b64-4e0a-8542-6e8439903141 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.698319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df9ead-4b64-4e0a-8542-6e8439903141 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.698590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df9ead-4b64-4e0a-8542-6e8439903141 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.709250 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df9ead-4b64-4e0a-8542-6e8439903141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-28df9ead-4b64-4e0a-8542-6e8439903141 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:13.717196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:13.718187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:13.719569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 287/571] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:13.720277 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df9ead-4b64-4e0a-8542-6e8439903141 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:13.744410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a833daf-e486-438f-9c51-977212576a79 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:13.777842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e99cf-76c7-4a25-9746-47e7b2819db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:13.778484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e99cf-76c7-4a25-9746-47e7b2819db3 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.778763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e99cf-76c7-4a25-9746-47e7b2819db3 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.779034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e99cf-76c7-4a25-9746-47e7b2819db3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.779858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e99cf-76c7-4a25-9746-47e7b2819db3 None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=c2bce9daec8c446d89eb6cc5950d6649) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:13.780168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e99cf-76c7-4a25-9746-47e7b2819db3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:13.782161 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e99cf-76c7-4a25-9746-47e7b2819db3 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:13.787950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 285/572] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:13 2022] GET /identity/v3/role_assignments?user.id=c2bce9daec8c446d89eb6cc5950d6649&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:13.788743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e99cf-76c7-4a25-9746-47e7b2819db3 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:13.824719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df9ead-4b64-4e0a-8542-6e8439903141 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:13.867414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649c5b9-6f39-4bc4-812c-bbcc2171772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:13.868330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649c5b9-6f39-4bc4-812c-bbcc2171772c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.868614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649c5b9-6f39-4bc4-812c-bbcc2171772c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.868889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649c5b9-6f39-4bc4-812c-bbcc2171772c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.881830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649c5b9-6f39-4bc4-812c-bbcc2171772c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:13.883308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649c5b9-6f39-4bc4-812c-bbcc2171772c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:13.894807 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-0649c5b9-6f39-4bc4-812c-bbcc2171772c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:13.905843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:13.906949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:13.907814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:13.907814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:13.907814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:13.907814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:13.907814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 288/573] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:13.908090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649c5b9-6f39-4bc4-812c-bbcc2171772c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:13.937079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e99cf-76c7-4a25-9746-47e7b2819db3 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:13.950890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:13.979727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104646b-8a34-4645-9d1c-ad6de203b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:13.979936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104646b-8a34-4645-9d1c-ad6de203b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:13.980119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104646b-8a34-4645-9d1c-ad6de203b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:13.980377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104646b-8a34-4645-9d1c-ad6de203b7a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:13.992205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104646b-8a34-4645-9d1c-ad6de203b7a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:13.993695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104646b-8a34-4645-9d1c-ad6de203b7a8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-c104646b-8a34-4645-9d1c-ad6de203b7a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:14.012124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:14.013014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:14.014232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:14.014232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:14.014232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:14.014232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:14.014232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 286/574] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:14.014499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104646b-8a34-4645-9d1c-ad6de203b7a8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.028572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649c5b9-6f39-4bc4-812c-bbcc2171772c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.067574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926700a9-3572-4eb5-b4c4-41b85d800877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.068391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926700a9-3572-4eb5-b4c4-41b85d800877 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.068673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926700a9-3572-4eb5-b4c4-41b85d800877 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.068946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926700a9-3572-4eb5-b4c4-41b85d800877 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.079344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926700a9-3572-4eb5-b4c4-41b85d800877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-926700a9-3572-4eb5-b4c4-41b85d800877 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:14.087184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:14.088106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:14.088919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 289/575] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:14.089608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926700a9-3572-4eb5-b4c4-41b85d800877 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.132320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104646b-8a34-4645-9d1c-ad6de203b7a8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.161650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:14.172740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b73c72-f371-44fe-ac16-92e0b5226eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.173538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b73c72-f371-44fe-ac16-92e0b5226eb2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.173848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b73c72-f371-44fe-ac16-92e0b5226eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.174136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b73c72-f371-44fe-ac16-92e0b5226eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.174871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b73c72-f371-44fe-ac16-92e0b5226eb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:14.175176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b73c72-f371-44fe-ac16-92e0b5226eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:14.177019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b73c72-f371-44fe-ac16-92e0b5226eb2 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.192204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 287/576] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:14.192204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b73c72-f371-44fe-ac16-92e0b5226eb2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.202283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:14.207395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926700a9-3572-4eb5-b4c4-41b85d800877 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.244034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae153ea2-e700-49db-a7b9-caad96e04dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.244825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae153ea2-e700-49db-a7b9-caad96e04dbf None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.245176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae153ea2-e700-49db-a7b9-caad96e04dbf None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.245518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae153ea2-e700-49db-a7b9-caad96e04dbf None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.246342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae153ea2-e700-49db-a7b9-caad96e04dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:14.246715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae153ea2-e700-49db-a7b9-caad96e04dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:14.248163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae153ea2-e700-49db-a7b9-caad96e04dbf None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.265270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 290/577] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:14.265270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae153ea2-e700-49db-a7b9-caad96e04dbf None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.303358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b73c72-f371-44fe-ac16-92e0b5226eb2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.338072 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d09a8-c133-47c4-a09b-3efd1ca7591b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.338734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d09a8-c133-47c4-a09b-3efd1ca7591b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.339011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d09a8-c133-47c4-a09b-3efd1ca7591b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.339284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d09a8-c133-47c4-a09b-3efd1ca7591b None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.339999 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d09a8-c133-47c4-a09b-3efd1ca7591b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:14.340303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d09a8-c133-47c4-a09b-3efd1ca7591b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:14.341727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d09a8-c133-47c4-a09b-3efd1ca7591b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.354591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 288/578] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:14.355455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d09a8-c133-47c4-a09b-3efd1ca7591b None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.374927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae153ea2-e700-49db-a7b9-caad96e04dbf None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.408355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:14.413290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ef994-e1d6-4c59-b485-d6676867dd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.413986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ef994-e1d6-4c59-b485-d6676867dd02 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.414272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ef994-e1d6-4c59-b485-d6676867dd02 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.414554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ef994-e1d6-4c59-b485-d6676867dd02 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.415441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ef994-e1d6-4c59-b485-d6676867dd02 None admin] RBAC: Adding query filter params (name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:14.415752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ef994-e1d6-4c59-b485-d6676867dd02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:14.417121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ef994-e1d6-4c59-b485-d6676867dd02 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.430203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 291/579] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:14.430203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ef994-e1d6-4c59-b485-d6676867dd02 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.455067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:14.468381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d09a8-c133-47c4-a09b-3efd1ca7591b None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.508020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14bb62-93c3-46d1-a087-3a36db1744d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.508821 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14bb62-93c3-46d1-a087-3a36db1744d7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.509101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14bb62-93c3-46d1-a087-3a36db1744d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.509376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14bb62-93c3-46d1-a087-3a36db1744d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.520898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14bb62-93c3-46d1-a087-3a36db1744d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:14.522583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14bb62-93c3-46d1-a087-3a36db1744d7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.540980 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ef994-e1d6-4c59-b485-d6676867dd02 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-5b14bb62-93c3-46d1-a087-3a36db1744d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:14.542337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:14.543339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:14.544438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:14.544438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:14.544438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:14.544438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:14.549717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 289/580] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:14.550018 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14bb62-93c3-46d1-a087-3a36db1744d7 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.584856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f392e-1ce3-43a4-8585-96c65c7194ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.585860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f392e-1ce3-43a4-8585-96c65c7194ba None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.586235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f392e-1ce3-43a4-8585-96c65c7194ba None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.586597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f392e-1ce3-43a4-8585-96c65c7194ba None admin] PATH_INFO: `/v3/users/glance` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.595646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f392e-1ce3-43a4-8585-96c65c7194ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:14.597155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f392e-1ce3-43a4-8585-96c65c7194ba None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-3d4f392e-1ce3-43a4-8585-96c65c7194ba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:14.612125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:14.613332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:14.614544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:14.614544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:14.614544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:14.614544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:14.614544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:14.614544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:14.614544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:14.614544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:14.614544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 19:40:14.614544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:14.619545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 292/581] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:14.620364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f392e-1ce3-43a4-8585-96c65c7194ba None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.660353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:14.664945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14bb62-93c3-46d1-a087-3a36db1744d7 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.700362 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a55f3-d88b-436c-b3aa-19ceef42bfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.701276 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a55f3-d88b-436c-b3aa-19ceef42bfd8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.701694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a55f3-d88b-436c-b3aa-19ceef42bfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.702074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a55f3-d88b-436c-b3aa-19ceef42bfd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.706212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:14.709502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a55f3-d88b-436c-b3aa-19ceef42bfd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:14.711087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a55f3-d88b-436c-b3aa-19ceef42bfd8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-714a55f3-d88b-436c-b3aa-19ceef42bfd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:14.726371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:14.728493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:14.729386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:14.729386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:14.729386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:14.729386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:14.729386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:14.729386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:14.729386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:14.729386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:14.729386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 290/582] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:14.729386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a55f3-d88b-436c-b3aa-19ceef42bfd8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.734351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f392e-1ce3-43a4-8585-96c65c7194ba None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.771581 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d05b3-c86e-4cfe-a14d-cac3f8024b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.772297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d05b3-c86e-4cfe-a14d-cac3f8024b90 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.772577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d05b3-c86e-4cfe-a14d-cac3f8024b90 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.772849 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d05b3-c86e-4cfe-a14d-cac3f8024b90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.783890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d05b3-c86e-4cfe-a14d-cac3f8024b90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:14.785354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d05b3-c86e-4cfe-a14d-cac3f8024b90 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6d0d05b3-c86e-4cfe-a14d-cac3f8024b90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:14.803074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:14.803974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:14.804921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:14.804921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:14.804921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:14.804921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:14.804921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 293/583] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:14.805191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d05b3-c86e-4cfe-a14d-cac3f8024b90 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.844856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a55f3-d88b-436c-b3aa-19ceef42bfd8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.881623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687bda0-77c0-499b-8189-ea55c9eb8256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.882374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687bda0-77c0-499b-8189-ea55c9eb8256 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.882661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687bda0-77c0-499b-8189-ea55c9eb8256 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.882933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687bda0-77c0-499b-8189-ea55c9eb8256 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.883658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687bda0-77c0-499b-8189-ea55c9eb8256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:14.883962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687bda0-77c0-499b-8189-ea55c9eb8256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:14.885340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687bda0-77c0-499b-8189-ea55c9eb8256 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.899523 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 291/584] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:14.900627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687bda0-77c0-499b-8189-ea55c9eb8256 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:14.914809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d05b3-c86e-4cfe-a14d-cac3f8024b90 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:14.928148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:14.949342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:14.957521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab2bf1-8363-44f3-a56b-f3e4f9b07aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:14.958326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab2bf1-8363-44f3-a56b-f3e4f9b07aa7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:14.958614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab2bf1-8363-44f3-a56b-f3e4f9b07aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:14.958893 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab2bf1-8363-44f3-a56b-f3e4f9b07aa7 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:14.959742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab2bf1-8363-44f3-a56b-f3e4f9b07aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:14.960077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab2bf1-8363-44f3-a56b-f3e4f9b07aa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:14.961872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab2bf1-8363-44f3-a56b-f3e4f9b07aa7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:14.981597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 294/585] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:14.982490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab2bf1-8363-44f3-a56b-f3e4f9b07aa7 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:15.024349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687bda0-77c0-499b-8189-ea55c9eb8256 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:15.059408 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef189a8b-a901-4eba-974a-72437da92a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:15.060061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef189a8b-a901-4eba-974a-72437da92a30 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:15.060342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef189a8b-a901-4eba-974a-72437da92a30 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:15.060617 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef189a8b-a901-4eba-974a-72437da92a30 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:15.061615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef189a8b-a901-4eba-974a-72437da92a30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:15.061724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef189a8b-a901-4eba-974a-72437da92a30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:15.063181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef189a8b-a901-4eba-974a-72437da92a30 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:15.075391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 292/586] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:15.076086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef189a8b-a901-4eba-974a-72437da92a30 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:15.092884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab2bf1-8363-44f3-a56b-f3e4f9b07aa7 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:15.128479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a8cd9-eb55-4901-8760-8792bbb3b7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:15.129132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a8cd9-eb55-4901-8760-8792bbb3b7ef None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:15.129402 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a8cd9-eb55-4901-8760-8792bbb3b7ef None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:15.129728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a8cd9-eb55-4901-8760-8792bbb3b7ef None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:15.130451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a8cd9-eb55-4901-8760-8792bbb3b7ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:15.130741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a8cd9-eb55-4901-8760-8792bbb3b7ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:15.132123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a8cd9-eb55-4901-8760-8792bbb3b7ef None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:15.148367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 295/587] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:15.148367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a8cd9-eb55-4901-8760-8792bbb3b7ef None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:15.169447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:15.190669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef189a8b-a901-4eba-974a-72437da92a30 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:15.196859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:15.229806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c6f4f-a3b2-4253-ac6c-06e84fe7c6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:15.230599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c6f4f-a3b2-4253-ac6c-06e84fe7c6fb None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:15.230876 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c6f4f-a3b2-4253-ac6c-06e84fe7c6fb None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:15.231345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c6f4f-a3b2-4253-ac6c-06e84fe7c6fb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:15.239357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c6f4f-a3b2-4253-ac6c-06e84fe7c6fb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:15.240966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c6f4f-a3b2-4253-ac6c-06e84fe7c6fb None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:15.251004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a8cd9-eb55-4901-8760-8792bbb3b7ef None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-103c6f4f-a3b2-4253-ac6c-06e84fe7c6fb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:15.256495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:15.257441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:15.258389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 293/588] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:15.258966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c6f4f-a3b2-4253-ac6c-06e84fe7c6fb None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:15.291628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d14b5-1405-4ad8-a56f-7712ba11802d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:15.292340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d14b5-1405-4ad8-a56f-7712ba11802d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:15.292619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d14b5-1405-4ad8-a56f-7712ba11802d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:15.292889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d14b5-1405-4ad8-a56f-7712ba11802d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:15.304945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d14b5-1405-4ad8-a56f-7712ba11802d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:15.306455 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d14b5-1405-4ad8-a56f-7712ba11802d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-773d14b5-1405-4ad8-a56f-7712ba11802d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:15.325996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:15.326976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:15.333239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:15.333239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:15.333239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:15.333239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:15.333239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 296/589] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:15.333239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d14b5-1405-4ad8-a56f-7712ba11802d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:15.369332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c6f4f-a3b2-4253-ac6c-06e84fe7c6fb None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:15.407221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985683f-d768-4f5d-8c85-de39b608514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:15.408107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985683f-d768-4f5d-8c85-de39b608514d None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:15.408386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985683f-d768-4f5d-8c85-de39b608514d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:15.408658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985683f-d768-4f5d-8c85-de39b608514d None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/ad06f31859f9413f9db6039ac2abf982/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:15.420760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:15.424517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985683f-d768-4f5d-8c85-de39b608514d None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=ad06f31859f9413f9db6039ac2abf982, role_id=0db2f13c4dbe42a4a0d8ffa0bb5633fd)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:15.426457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985683f-d768-4f5d-8c85-de39b608514d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:15.441767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 294/590] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:40:15 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/ad06f31859f9413f9db6039ac2abf982/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:40:15.441767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985683f-d768-4f5d-8c85-de39b608514d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:15.444647 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d14b5-1405-4ad8-a56f-7712ba11802d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:15.446669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:15.490256 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafd6c1-a663-49ff-9090-2a4a32a203ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:15.491067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafd6c1-a663-49ff-9090-2a4a32a203ac None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:15.491346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafd6c1-a663-49ff-9090-2a4a32a203ac None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:15.491618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafd6c1-a663-49ff-9090-2a4a32a203ac None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:15.500157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafd6c1-a663-49ff-9090-2a4a32a203ac None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:15.501782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafd6c1-a663-49ff-9090-2a4a32a203ac None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-ffafd6c1-a663-49ff-9090-2a4a32a203ac None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:15.519247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:15.520272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:15.521202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:15.521202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:15.521202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:15.521202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:15.521202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:15.521202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:15.521202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:15.521202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:15.521202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 19:40:15.521202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:15.525814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 297/591] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:15.526782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafd6c1-a663-49ff-9090-2a4a32a203ac None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:15.590996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985683f-d768-4f5d-8c85-de39b608514d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:15.638856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecf50a-be35-43ff-9fb4-57152cfb385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:15.640083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecf50a-be35-43ff-9fb4-57152cfb385c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:15.640572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecf50a-be35-43ff-9fb4-57152cfb385c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:15.640989 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecf50a-be35-43ff-9fb4-57152cfb385c None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:15.642328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecf50a-be35-43ff-9fb4-57152cfb385c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:15.642998 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecf50a-be35-43ff-9fb4-57152cfb385c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:15.644491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecf50a-be35-43ff-9fb4-57152cfb385c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:15.649673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafd6c1-a663-49ff-9090-2a4a32a203ac None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:15.666139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 295/592] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 1 19:40:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:15.666922 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecf50a-be35-43ff-9fb4-57152cfb385c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:15.693294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371b089-caba-4fa5-a5f4-77bae38d00dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:15.694393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371b089-caba-4fa5-a5f4-77bae38d00dc None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:15.694911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371b089-caba-4fa5-a5f4-77bae38d00dc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:15.695363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371b089-caba-4fa5-a5f4-77bae38d00dc None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:15.696265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371b089-caba-4fa5-a5f4-77bae38d00dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:15.696745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371b089-caba-4fa5-a5f4-77bae38d00dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:15.698341 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371b089-caba-4fa5-a5f4-77bae38d00dc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:15.711750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 298/593] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:15.712929 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371b089-caba-4fa5-a5f4-77bae38d00dc None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:15.741898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:15.747498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:15.810262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecf50a-be35-43ff-9fb4-57152cfb385c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:15.832925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371b089-caba-4fa5-a5f4-77bae38d00dc None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:15.858755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba67af-a00c-4fc0-819a-574aaeeec192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:15.859465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba67af-a00c-4fc0-819a-574aaeeec192 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:15.859750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba67af-a00c-4fc0-819a-574aaeeec192 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:15.860025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba67af-a00c-4fc0-819a-574aaeeec192 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:15.861115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba67af-a00c-4fc0-819a-574aaeeec192 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:15.861425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba67af-a00c-4fc0-819a-574aaeeec192 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:15.863582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba67af-a00c-4fc0-819a-574aaeeec192 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:15.871323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbc07e-e67d-4913-a8f0-2cebc5eb9243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:15.872433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbc07e-e67d-4913-a8f0-2cebc5eb9243 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:15.872914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbc07e-e67d-4913-a8f0-2cebc5eb9243 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:15.873445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbc07e-e67d-4913-a8f0-2cebc5eb9243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:15.885113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbc07e-e67d-4913-a8f0-2cebc5eb9243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:15.889610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 296/594] 10.222.0.66 () {56 vars in 1139 bytes} [Fri Jul 1 19:40:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:15.889610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba67af-a00c-4fc0-819a-574aaeeec192 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:15.891691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbc07e-e67d-4913-a8f0-2cebc5eb9243 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-67cbc07e-e67d-4913-a8f0-2cebc5eb9243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:15.912972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:15.913928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:15.914797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:15.914797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:15.914797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:15.914797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:15.914797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 299/595] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:15.915478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbc07e-e67d-4913-a8f0-2cebc5eb9243 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:16.002915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:16.015216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba67af-a00c-4fc0-819a-574aaeeec192 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:16.025218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:16.025956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbc07e-e67d-4913-a8f0-2cebc5eb9243 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:16.057480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0c164-496f-4669-a0eb-ecd06c6ca41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:16.059455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0c164-496f-4669-a0eb-ecd06c6ca41b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.059807 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0c164-496f-4669-a0eb-ecd06c6ca41b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:16.060090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0c164-496f-4669-a0eb-ecd06c6ca41b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:16.069627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0c164-496f-4669-a0eb-ecd06c6ca41b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:16.071063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1d29c-6208-4f9b-b973-1e21434f84a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:16.071742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0c164-496f-4669-a0eb-ecd06c6ca41b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:16.071845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1d29c-6208-4f9b-b973-1e21434f84a4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.072134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1d29c-6208-4f9b-b973-1e21434f84a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:16.072404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1d29c-6208-4f9b-b973-1e21434f84a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:16.084173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1d29c-6208-4f9b-b973-1e21434f84a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:16.085631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1d29c-6208-4f9b-b973-1e21434f84a4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-4ab0c164-496f-4669-a0eb-ecd06c6ca41b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:16.086574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:16.087398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:16.088295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:16.088295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:16.088295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:16.088295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:16.088295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:16.088295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:16.088295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:16.088295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:16.088295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 297/596] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:16.089327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704b5e2-4b80-45c6-a49b-9ec83cfd0617 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.089434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704b5e2-4b80-45c6-a49b-9ec83cfd0617 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:16.089646 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704b5e2-4b80-45c6-a49b-9ec83cfd0617 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:16.091043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 298/597] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:16.091657 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704b5e2-4b80-45c6-a49b-9ec83cfd0617 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-d6e1d29c-6208-4f9b-b973-1e21434f84a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:16.113644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:16.114674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:16.115592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:16.115592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:16.115592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:16.115592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:16.115592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 300/598] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:16.115876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1d29c-6208-4f9b-b973-1e21434f84a4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:16.216538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704b5e2-4b80-45c6-a49b-9ec83cfd0617 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:16.243635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1d29c-6208-4f9b-b973-1e21434f84a4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:16.258302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40d313-f093-47b0-82e1-6cd35096b821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:16.258960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40d313-f093-47b0-82e1-6cd35096b821 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.259268 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40d313-f093-47b0-82e1-6cd35096b821 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:16.259504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40d313-f093-47b0-82e1-6cd35096b821 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:16.260217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40d313-f093-47b0-82e1-6cd35096b821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:16.260759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40d313-f093-47b0-82e1-6cd35096b821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:16.261915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40d313-f093-47b0-82e1-6cd35096b821 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:16.279715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 299/599] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:16.279715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262457ab-2095-494d-b6d6-6619fd767eb0 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.279715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262457ab-2095-494d-b6d6-6619fd767eb0 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:16.279715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262457ab-2095-494d-b6d6-6619fd767eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:16.283103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:16.296390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8673c6-ed92-45c9-b8b5-35b01201b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:16.297695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8673c6-ed92-45c9-b8b5-35b01201b6df None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.298068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8673c6-ed92-45c9-b8b5-35b01201b6df None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:16.298413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8673c6-ed92-45c9-b8b5-35b01201b6df None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:16.299213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8673c6-ed92-45c9-b8b5-35b01201b6df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:16.299521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8673c6-ed92-45c9-b8b5-35b01201b6df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:16.300956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8673c6-ed92-45c9-b8b5-35b01201b6df None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:16.302525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-262457ab-2095-494d-b6d6-6619fd767eb0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:16.304677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-262457ab-2095-494d-b6d6-6619fd767eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:16.329717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 301/600] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:16.329717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8673c6-ed92-45c9-b8b5-35b01201b6df None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:16.337220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:16.400842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 300/601] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:16.410581 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262457ab-2095-494d-b6d6-6619fd767eb0 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:16.465995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8673c6-ed92-45c9-b8b5-35b01201b6df None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:16.515236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29362fd-3624-44f8-a6c6-22395af3ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:16.516504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29362fd-3624-44f8-a6c6-22395af3ca1f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.516892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29362fd-3624-44f8-a6c6-22395af3ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:16.517329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29362fd-3624-44f8-a6c6-22395af3ca1f None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:16.518618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29362fd-3624-44f8-a6c6-22395af3ca1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:16.519076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29362fd-3624-44f8-a6c6-22395af3ca1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:16.520502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29362fd-3624-44f8-a6c6-22395af3ca1f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:16.538037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 302/602] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:16.539077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29362fd-3624-44f8-a6c6-22395af3ca1f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:16.549471 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262457ab-2095-494d-b6d6-6619fd767eb0 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:16.602708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13a3ad-c17f-4370-acb6-2fc308167e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:16.603425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13a3ad-c17f-4370-acb6-2fc308167e63 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.604064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13a3ad-c17f-4370-acb6-2fc308167e63 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:16.604064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13a3ad-c17f-4370-acb6-2fc308167e63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:16.611621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:16.611860 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13a3ad-c17f-4370-acb6-2fc308167e63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:16.613407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13a3ad-c17f-4370-acb6-2fc308167e63 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-af13a3ad-c17f-4370-acb6-2fc308167e63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:16.629693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:16.630805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:16.632625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:16.632625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:16.632625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:16.632625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:16.632625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:16.632625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:16.632625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:16.632625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:16.632625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 301/603] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:16.634438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13a3ad-c17f-4370-acb6-2fc308167e63 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:16.663593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:16.698035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29362fd-3624-44f8-a6c6-22395af3ca1f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:16.745472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76fdc3-aa6b-4f83-ab6a-93c286cc224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:16.745472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76fdc3-aa6b-4f83-ab6a-93c286cc224c None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.745984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76fdc3-aa6b-4f83-ab6a-93c286cc224c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:16.746500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76fdc3-aa6b-4f83-ab6a-93c286cc224c None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/c022224b03a54ab489d95502c63a25dd/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:16.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76fdc3-aa6b-4f83-ab6a-93c286cc224c None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=c022224b03a54ab489d95502c63a25dd, role_id=0db2f13c4dbe42a4a0d8ffa0bb5633fd)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:16.764533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76fdc3-aa6b-4f83-ab6a-93c286cc224c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:16.772668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13a3ad-c17f-4370-acb6-2fc308167e63 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:16.779282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 303/604] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:40:16 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/c022224b03a54ab489d95502c63a25dd/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:40:16.779887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76fdc3-aa6b-4f83-ab6a-93c286cc224c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:16.842194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6430d-baac-44d1-ac94-c9597eebc165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:16.843416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6430d-baac-44d1-ac94-c9597eebc165 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.845486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6430d-baac-44d1-ac94-c9597eebc165 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:16.846035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6430d-baac-44d1-ac94-c9597eebc165 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:16.862552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6430d-baac-44d1-ac94-c9597eebc165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-d6f6430d-baac-44d1-ac94-c9597eebc165 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:16.874176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:16.876365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:16.877209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:16.884391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 302/605] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:16.885365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6430d-baac-44d1-ac94-c9597eebc165 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:16.935590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:16.948670 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76fdc3-aa6b-4f83-ab6a-93c286cc224c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:16.998816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d27975-e314-4ea8-a7ff-d05fdac53ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:16.999659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27975-e314-4ea8-a7ff-d05fdac53ea9 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:16.999969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27975-e314-4ea8-a7ff-d05fdac53ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.000252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27975-e314-4ea8-a7ff-d05fdac53ea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.003210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:17.010312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d27975-e314-4ea8-a7ff-d05fdac53ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:17.012179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d27975-e314-4ea8-a7ff-d05fdac53ea9 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-99d27975-e314-4ea8-a7ff-d05fdac53ea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:17.028131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:17.029248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:17.030197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:17.030197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:17.030197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:17.030197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:17.030197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:17.030197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:17.030197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:17.030197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:17.030197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 304/606] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:17.030671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d27975-e314-4ea8-a7ff-d05fdac53ea9 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:17.037343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6430d-baac-44d1-ac94-c9597eebc165 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:17.072121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578223e-54f5-42ae-ab26-b65b918473d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:17.073039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578223e-54f5-42ae-ab26-b65b918473d9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:17.073415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578223e-54f5-42ae-ab26-b65b918473d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.073832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578223e-54f5-42ae-ab26-b65b918473d9 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.074793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578223e-54f5-42ae-ab26-b65b918473d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:17.075229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578223e-54f5-42ae-ab26-b65b918473d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:17.076868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578223e-54f5-42ae-ab26-b65b918473d9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:17.091017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 303/607] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:17.092162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578223e-54f5-42ae-ab26-b65b918473d9 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:17.170891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d27975-e314-4ea8-a7ff-d05fdac53ea9 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:17.215674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578223e-54f5-42ae-ab26-b65b918473d9 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:17.217690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532407e8-4f0b-46b1-b350-dddaaef89fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:17.220024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532407e8-4f0b-46b1-b350-dddaaef89fe0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:17.220466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532407e8-4f0b-46b1-b350-dddaaef89fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.220890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532407e8-4f0b-46b1-b350-dddaaef89fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.221900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532407e8-4f0b-46b1-b350-dddaaef89fe0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:17.222376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532407e8-4f0b-46b1-b350-dddaaef89fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:17.223923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532407e8-4f0b-46b1-b350-dddaaef89fe0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:17.238545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 305/608] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:17.240340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532407e8-4f0b-46b1-b350-dddaaef89fe0 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:17.255657 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63726e-7980-40a4-9874-866bb03d23d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:17.256499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63726e-7980-40a4-9874-866bb03d23d4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:17.256897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63726e-7980-40a4-9874-866bb03d23d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.257265 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63726e-7980-40a4-9874-866bb03d23d4 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.258552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63726e-7980-40a4-9874-866bb03d23d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:17.258942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63726e-7980-40a4-9874-866bb03d23d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:17.260526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63726e-7980-40a4-9874-866bb03d23d4 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:17.264162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:17.267762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:17.282028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 304/609] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:17.283174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63726e-7980-40a4-9874-866bb03d23d4 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:17.378555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532407e8-4f0b-46b1-b350-dddaaef89fe0 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:17.405084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63726e-7980-40a4-9874-866bb03d23d4 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:17.427487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa74b4e-3a5e-4d8c-8a91-8c366d5895de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:17.429149 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74b4e-3a5e-4d8c-8a91-8c366d5895de None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:17.429860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74b4e-3a5e-4d8c-8a91-8c366d5895de None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.430314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74b4e-3a5e-4d8c-8a91-8c366d5895de None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/a4b0e6ea57af454cbc8f5bd180bc052e/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.444118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c257617-82b2-4faf-bace-33b2f1ba5ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:17.445174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c257617-82b2-4faf-bace-33b2f1ba5ae1 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:17.445544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c257617-82b2-4faf-bace-33b2f1ba5ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.446083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c257617-82b2-4faf-bace-33b2f1ba5ae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.449558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa74b4e-3a5e-4d8c-8a91-8c366d5895de None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=a4b0e6ea57af454cbc8f5bd180bc052e, role_id=0db2f13c4dbe42a4a0d8ffa0bb5633fd)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:17.451192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa74b4e-3a5e-4d8c-8a91-8c366d5895de None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:17.461630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c257617-82b2-4faf-bace-33b2f1ba5ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:17.464422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c257617-82b2-4faf-bace-33b2f1ba5ae1 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:17.478309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 306/610] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:40:17 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/a4b0e6ea57af454cbc8f5bd180bc052e/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:40:17.478825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa74b4e-3a5e-4d8c-8a91-8c366d5895de None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-4c257617-82b2-4faf-bace-33b2f1ba5ae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:17.501787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:17.502739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:17.503613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:17.503613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:17.503613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:17.503613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:17.509145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 305/611] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:17.513304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c257617-82b2-4faf-bace-33b2f1ba5ae1 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:17.624739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:17.631418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:17.677812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa74b4e-3a5e-4d8c-8a91-8c366d5895de None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:17.689533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c257617-82b2-4faf-bace-33b2f1ba5ae1 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:17.723601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fe5b7-ac0a-462f-870f-425bca12f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:17.724604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fe5b7-ac0a-462f-870f-425bca12f97a None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:17.724883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fe5b7-ac0a-462f-870f-425bca12f97a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.725156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fe5b7-ac0a-462f-870f-425bca12f97a None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/370f1c0f7af04a56bdf8fa452550380a/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.732031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ca702-9f98-4866-85c7-f1c70708d56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:17.732697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ca702-9f98-4866-85c7-f1c70708d56a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:17.733028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ca702-9f98-4866-85c7-f1c70708d56a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.733307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ca702-9f98-4866-85c7-f1c70708d56a None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.734069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ca702-9f98-4866-85c7-f1c70708d56a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:17.735328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ca702-9f98-4866-85c7-f1c70708d56a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:17.736732 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ca702-9f98-4866-85c7-f1c70708d56a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:17.741845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fe5b7-ac0a-462f-870f-425bca12f97a None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=370f1c0f7af04a56bdf8fa452550380a, role_id=0db2f13c4dbe42a4a0d8ffa0bb5633fd)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:17.743330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fe5b7-ac0a-462f-870f-425bca12f97a None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:17.749375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 306/612] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:17.751014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef119d85-e94e-43a4-bc16-ddf99c662792 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:17.751307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef119d85-e94e-43a4-bc16-ddf99c662792 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.751584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef119d85-e94e-43a4-bc16-ddf99c662792 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.753104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 307/613] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:17.755937 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef119d85-e94e-43a4-bc16-ddf99c662792 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:17.765485 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 307/614] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:40:17 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/370f1c0f7af04a56bdf8fa452550380a/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:40:17.767382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35713b9-d627-4975-9907-3174291822e3 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:17.767566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35713b9-d627-4975-9907-3174291822e3 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.767931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35713b9-d627-4975-9907-3174291822e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.788445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f35713b9-d627-4975-9907-3174291822e3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:17.789847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f35713b9-d627-4975-9907-3174291822e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:17.870955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef119d85-e94e-43a4-bc16-ddf99c662792 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:17.872702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 308/615] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:17.882258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:17.901006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35713b9-d627-4975-9907-3174291822e3 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:17.910834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fe8ca-7c1d-4bd0-b5b0-827f0f7ded7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:17.911778 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fe8ca-7c1d-4bd0-b5b0-827f0f7ded7c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:17.912132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fe8ca-7c1d-4bd0-b5b0-827f0f7ded7c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:17.912476 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fe8ca-7c1d-4bd0-b5b0-827f0f7ded7c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:17.920954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fe8ca-7c1d-4bd0-b5b0-827f0f7ded7c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:17.923088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fe8ca-7c1d-4bd0-b5b0-827f0f7ded7c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-b59fe8ca-7c1d-4bd0-b5b0-827f0f7ded7c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:17.939093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:17.940223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:17.941346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:17.941346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:17.941346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:17.941346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:17.941346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:17.941346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:17.941346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:17.941346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:17.941346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:17.941346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:17.947347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 308/616] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:17.948698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fe8ca-7c1d-4bd0-b5b0-827f0f7ded7c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:17.961067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:18.025185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35713b9-d627-4975-9907-3174291822e3 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:18.064139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fe8ca-7c1d-4bd0-b5b0-827f0f7ded7c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:18.066139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61473f-5da4-4787-9ad9-47dadef5c957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:18.067000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61473f-5da4-4787-9ad9-47dadef5c957 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:18.067282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61473f-5da4-4787-9ad9-47dadef5c957 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:18.067550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61473f-5da4-4787-9ad9-47dadef5c957 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:18.083450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61473f-5da4-4787-9ad9-47dadef5c957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-af61473f-5da4-4787-9ad9-47dadef5c957 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:18.093623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:18.095105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:18.095907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:18.099014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 309/617] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:18.105711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61473f-5da4-4787-9ad9-47dadef5c957 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:18.123071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454ffa2-e166-4c46-80dd-d40dba1a97fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:18.124202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454ffa2-e166-4c46-80dd-d40dba1a97fd None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:18.126593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454ffa2-e166-4c46-80dd-d40dba1a97fd None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:18.127022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454ffa2-e166-4c46-80dd-d40dba1a97fd None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:18.128119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454ffa2-e166-4c46-80dd-d40dba1a97fd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:18.128784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454ffa2-e166-4c46-80dd-d40dba1a97fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:18.131443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454ffa2-e166-4c46-80dd-d40dba1a97fd None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:18.168491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 309/618] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 1 19:40:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:18.175563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454ffa2-e166-4c46-80dd-d40dba1a97fd None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:18.230086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:18.248072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61473f-5da4-4787-9ad9-47dadef5c957 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:18.270051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:18.285058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297081ef-6ec2-4193-bf66-246a7125eaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:18.286364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297081ef-6ec2-4193-bf66-246a7125eaae None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:18.286857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297081ef-6ec2-4193-bf66-246a7125eaae None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:18.287399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297081ef-6ec2-4193-bf66-246a7125eaae None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:18.288357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297081ef-6ec2-4193-bf66-246a7125eaae None admin] RBAC: Adding query filter params (name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:18.288873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297081ef-6ec2-4193-bf66-246a7125eaae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:18.290566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297081ef-6ec2-4193-bf66-246a7125eaae None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:18.300247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 310/619] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:18.305194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297081ef-6ec2-4193-bf66-246a7125eaae None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:18.363317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454ffa2-e166-4c46-80dd-d40dba1a97fd None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:18.413105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297081ef-6ec2-4193-bf66-246a7125eaae None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:18.451674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128c92a-6438-440b-911a-e498347e0a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:18.453811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128c92a-6438-440b-911a-e498347e0a2f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:18.454329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128c92a-6438-440b-911a-e498347e0a2f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:18.454874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128c92a-6438-440b-911a-e498347e0a2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:18.466613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128c92a-6438-440b-911a-e498347e0a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:18.468387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128c92a-6438-440b-911a-e498347e0a2f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:18.478295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277546b2-47e3-4c00-9f7d-21f81e27e173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:18.479102 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277546b2-47e3-4c00-9f7d-21f81e27e173 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:18.479395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277546b2-47e3-4c00-9f7d-21f81e27e173 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:18.479672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277546b2-47e3-4c00-9f7d-21f81e27e173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-b128c92a-6438-440b-911a-e498347e0a2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:18.488509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:18.489390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:18.490316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:18.490316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:18.490316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:18.490316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:18.492686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 311/620] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:18.498896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128c92a-6438-440b-911a-e498347e0a2f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:18.526462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277546b2-47e3-4c00-9f7d-21f81e27e173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:18.527936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277546b2-47e3-4c00-9f7d-21f81e27e173 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:18.528880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-277546b2-47e3-4c00-9f7d-21f81e27e173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:18.571179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:18.572087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:18.572978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:18.572978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:18.572978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:18.572978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:18.578083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 310/621] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:18.586527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277546b2-47e3-4c00-9f7d-21f81e27e173 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:18.612599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128c92a-6438-440b-911a-e498347e0a2f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:18.649292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39fb9a-276d-45ce-b130-680959949eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:18.650473 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39fb9a-276d-45ce-b130-680959949eb6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:18.650764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39fb9a-276d-45ce-b130-680959949eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:18.651047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39fb9a-276d-45ce-b130-680959949eb6 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:18.651784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39fb9a-276d-45ce-b130-680959949eb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:18.652096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39fb9a-276d-45ce-b130-680959949eb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:18.653622 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39fb9a-276d-45ce-b130-680959949eb6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:18.668212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 312/622] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:18.673723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39fb9a-276d-45ce-b130-680959949eb6 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:18.778074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:18.784609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39fb9a-276d-45ce-b130-680959949eb6 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:18.824883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9176ed8-d158-4ea2-8dd2-41ca474bc211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:18.825729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9176ed8-d158-4ea2-8dd2-41ca474bc211 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:18.826009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9176ed8-d158-4ea2-8dd2-41ca474bc211 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:18.826282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9176ed8-d158-4ea2-8dd2-41ca474bc211 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:18.834897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9176ed8-d158-4ea2-8dd2-41ca474bc211 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:18.837664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9176ed8-d158-4ea2-8dd2-41ca474bc211 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:18.853958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-b9176ed8-d158-4ea2-8dd2-41ca474bc211 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:18.861162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:18.862114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:18.863055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:18.863055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:18.863055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:18.863055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:18.863055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:18.863055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:18.863055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:18.863055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:18.863055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:18.863055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:18.871521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 313/623] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 1 19:40:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:18.877436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9176ed8-d158-4ea2-8dd2-41ca474bc211 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:18.885771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277546b2-47e3-4c00-9f7d-21f81e27e173 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:18.925763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd6d60-9917-483b-a6d6-aa6a6fd442ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:18.926849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd6d60-9917-483b-a6d6-aa6a6fd442ab None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:18.927521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd6d60-9917-483b-a6d6-aa6a6fd442ab None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:18.928649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd6d60-9917-483b-a6d6-aa6a6fd442ab None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:18.929790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd6d60-9917-483b-a6d6-aa6a6fd442ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:18.931665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd6d60-9917-483b-a6d6-aa6a6fd442ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:18.933263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd6d60-9917-483b-a6d6-aa6a6fd442ab None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:18.957155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 311/624] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:18.961128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd6d60-9917-483b-a6d6-aa6a6fd442ab None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:19.071855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd6d60-9917-483b-a6d6-aa6a6fd442ab None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:19.108737 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1400c9c-374a-4e55-a4f7-39049f8afbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:19.110574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400c9c-374a-4e55-a4f7-39049f8afbfd None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:19.111076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400c9c-374a-4e55-a4f7-39049f8afbfd None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:19.111559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400c9c-374a-4e55-a4f7-39049f8afbfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:19.118898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1400c9c-374a-4e55-a4f7-39049f8afbfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:19.120598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1400c9c-374a-4e55-a4f7-39049f8afbfd None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:19.127469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-c1400c9c-374a-4e55-a4f7-39049f8afbfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:19.138882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:19.141923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:19.142861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:19.142861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:19.142861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:19.142861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:19.142861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:19.142861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:19.142861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:19.142861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:19.143328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9176ed8-d158-4ea2-8dd2-41ca474bc211 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:19.144814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 312/625] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:19.155888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1400c9c-374a-4e55-a4f7-39049f8afbfd None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:19.196682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead578a-1778-47b4-b8d7-89858e5759bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:19.197351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead578a-1778-47b4-b8d7-89858e5759bc None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:19.197662 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead578a-1778-47b4-b8d7-89858e5759bc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:19.197943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead578a-1778-47b4-b8d7-89858e5759bc None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:19.199006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead578a-1778-47b4-b8d7-89858e5759bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:19.199131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead578a-1778-47b4-b8d7-89858e5759bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:19.200507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead578a-1778-47b4-b8d7-89858e5759bc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:19.215372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:19.221246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 314/626] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 1 19:40:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:19.222740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadb56a-8072-4972-a7f0-356e5c772234 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:19.223031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadb56a-8072-4972-a7f0-356e5c772234 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:19.223308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadb56a-8072-4972-a7f0-356e5c772234 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:19.228681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 315/627] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:19.228681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadb56a-8072-4972-a7f0-356e5c772234 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:19.280348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1400c9c-374a-4e55-a4f7-39049f8afbfd None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:19.316208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98c257-0e6d-4200-ac12-42111a54bb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:19.317239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98c257-0e6d-4200-ac12-42111a54bb58 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:19.317746 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98c257-0e6d-4200-ac12-42111a54bb58 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:19.318215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98c257-0e6d-4200-ac12-42111a54bb58 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:19.319189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98c257-0e6d-4200-ac12-42111a54bb58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:19.319705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98c257-0e6d-4200-ac12-42111a54bb58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:19.321342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98c257-0e6d-4200-ac12-42111a54bb58 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:19.335004 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 313/628] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:19.336512 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b17641-4d45-4671-90b7-a98708222c6a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:19.336997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b17641-4d45-4671-90b7-a98708222c6a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:19.337469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b17641-4d45-4671-90b7-a98708222c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:19.340039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadb56a-8072-4972-a7f0-356e5c772234 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:19.354082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e7b17641-4d45-4671-90b7-a98708222c6a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:19.355585 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e7b17641-4d45-4671-90b7-a98708222c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:19.377778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e5205-91bd-4287-90d7-a81654c4c454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:19.378428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e5205-91bd-4287-90d7-a81654c4c454 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:19.378765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e5205-91bd-4287-90d7-a81654c4c454 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:19.378977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e5205-91bd-4287-90d7-a81654c4c454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:19.418037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:19.438094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 314/629] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:19.439254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b17641-4d45-4671-90b7-a98708222c6a None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:19.465678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e5205-91bd-4287-90d7-a81654c4c454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:19.467836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e5205-91bd-4287-90d7-a81654c4c454 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:19.482032 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:19.548342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b17641-4d45-4671-90b7-a98708222c6a None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:19.584163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ca3e2-406c-4017-80b1-bec1bd7f35be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:19.585260 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca3e2-406c-4017-80b1-bec1bd7f35be None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:19.585573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca3e2-406c-4017-80b1-bec1bd7f35be None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:19.585958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca3e2-406c-4017-80b1-bec1bd7f35be None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/c2bce9daec8c446d89eb6cc5950d6649/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:19.589180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 316/630] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:19.590196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e5205-91bd-4287-90d7-a81654c4c454 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:19.601379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ca3e2-406c-4017-80b1-bec1bd7f35be None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=c2bce9daec8c446d89eb6cc5950d6649, role_id=0db2f13c4dbe42a4a0d8ffa0bb5633fd)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:19.602857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ca3e2-406c-4017-80b1-bec1bd7f35be None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:19.618852 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 315/631] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:40:19 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/c2bce9daec8c446d89eb6cc5950d6649/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:40:19.619854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ca3e2-406c-4017-80b1-bec1bd7f35be None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:19.668885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:19.708798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e5205-91bd-4287-90d7-a81654c4c454 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:19.710813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:19.759115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4c957-2258-4b52-a220-e9965a4f2756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:19.759990 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4c957-2258-4b52-a220-e9965a4f2756 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:19.760272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4c957-2258-4b52-a220-e9965a4f2756 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:19.760548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4c957-2258-4b52-a220-e9965a4f2756 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:19.787509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4c957-2258-4b52-a220-e9965a4f2756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-74f4c957-2258-4b52-a220-e9965a4f2756 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:19.796763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:19.798291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:19.799152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 317/632] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:19.804606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4c957-2258-4b52-a220-e9965a4f2756 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:19.832516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ca3e2-406c-4017-80b1-bec1bd7f35be None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:19.888968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28839a-193b-4a6f-84a5-49b39ef3fac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:19.892269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28839a-193b-4a6f-84a5-49b39ef3fac9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:19.892559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28839a-193b-4a6f-84a5-49b39ef3fac9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:19.892837 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28839a-193b-4a6f-84a5-49b39ef3fac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:19.917359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28839a-193b-4a6f-84a5-49b39ef3fac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:19.919101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28839a-193b-4a6f-84a5-49b39ef3fac9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-5b28839a-193b-4a6f-84a5-49b39ef3fac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:19.954996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:19.955955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:19.956819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:19.956819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:19.956819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:19.956819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:19.956819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 316/633] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:19.963768 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28839a-193b-4a6f-84a5-49b39ef3fac9 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:19.966100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4c957-2258-4b52-a220-e9965a4f2756 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:20.029233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d1851-1fe0-4c98-9938-e6caea01a941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:20.030101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d1851-1fe0-4c98-9938-e6caea01a941 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.030381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d1851-1fe0-4c98-9938-e6caea01a941 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.032076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d1851-1fe0-4c98-9938-e6caea01a941 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.033126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d1851-1fe0-4c98-9938-e6caea01a941 None admin] RBAC: Adding query filter params (name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:20.033434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d1851-1fe0-4c98-9938-e6caea01a941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:20.034871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d1851-1fe0-4c98-9938-e6caea01a941 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:20.046065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 318/634] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:20.048382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a5d10-fb2b-4f54-91f1-a5021b6fca9b None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.048667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a5d10-fb2b-4f54-91f1-a5021b6fca9b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.048935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a5d10-fb2b-4f54-91f1-a5021b6fca9b None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.050941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 319/635] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:20.054427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a5d10-fb2b-4f54-91f1-a5021b6fca9b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:20.106923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:20.141686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:20.158583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28839a-193b-4a6f-84a5-49b39ef3fac9 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:20.231273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b23249-913a-4a14-ab69-6fcbae88153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:20.232483 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b23249-913a-4a14-ab69-6fcbae88153b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.233283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b23249-913a-4a14-ab69-6fcbae88153b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.234057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b23249-913a-4a14-ab69-6fcbae88153b None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.235533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b23249-913a-4a14-ab69-6fcbae88153b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:20.236190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b23249-913a-4a14-ab69-6fcbae88153b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:20.238340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b23249-913a-4a14-ab69-6fcbae88153b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:20.241507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a5d10-fb2b-4f54-91f1-a5021b6fca9b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:20.258464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 317/636] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:20.260644 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e1c34-f700-4001-9e4d-1db0672ef015 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.261026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e1c34-f700-4001-9e4d-1db0672ef015 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.261377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e1c34-f700-4001-9e4d-1db0672ef015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.278971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-bc5e1c34-f700-4001-9e4d-1db0672ef015 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:20.280253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-bc5e1c34-f700-4001-9e4d-1db0672ef015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:20.288301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42015c0c-6ace-48f8-9232-f99af47316a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:20.289460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015c0c-6ace-48f8-9232-f99af47316a7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.289950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015c0c-6ace-48f8-9232-f99af47316a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.290428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015c0c-6ace-48f8-9232-f99af47316a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.301984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42015c0c-6ace-48f8-9232-f99af47316a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:20.303807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42015c0c-6ace-48f8-9232-f99af47316a7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-42015c0c-6ace-48f8-9232-f99af47316a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:20.322950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:20.323885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:20.324772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:20.324772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:20.324772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:20.324772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:20.328458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 320/637] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:20.328458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e014fb-6e2c-4796-b881-bf397cacbd9a None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.328458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e014fb-6e2c-4796-b881-bf397cacbd9a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.328458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e014fb-6e2c-4796-b881-bf397cacbd9a None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.330080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 321/638] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:20.330968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e014fb-6e2c-4796-b881-bf397cacbd9a None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:20.352787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 318/639] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:20.353773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e1c34-f700-4001-9e4d-1db0672ef015 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:20.443010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:20.451174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:20.467971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e1c34-f700-4001-9e4d-1db0672ef015 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:20.487749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e014fb-6e2c-4796-b881-bf397cacbd9a None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:20.518302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d207a68-33b8-4587-82a7-8d05baceeec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:20.519054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207a68-33b8-4587-82a7-8d05baceeec7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.519330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207a68-33b8-4587-82a7-8d05baceeec7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.519692 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207a68-33b8-4587-82a7-8d05baceeec7 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.520305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207a68-33b8-4587-82a7-8d05baceeec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:20.520837 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207a68-33b8-4587-82a7-8d05baceeec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:20.521981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207a68-33b8-4587-82a7-8d05baceeec7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:20.533230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c9082-4fc6-438d-a485-897fd1599086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:20.534217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 319/640] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:20.534340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c9082-4fc6-438d-a485-897fd1599086 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.534558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c9082-4fc6-438d-a485-897fd1599086 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.534906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c9082-4fc6-438d-a485-897fd1599086 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.539355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65b89f-e65e-4832-803c-19f72b5a1595 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.539640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65b89f-e65e-4832-803c-19f72b5a1595 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.539914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65b89f-e65e-4832-803c-19f72b5a1595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.542395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c9082-4fc6-438d-a485-897fd1599086 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:20.543837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c9082-4fc6-438d-a485-897fd1599086 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:20.556401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-6e65b89f-e65e-4832-803c-19f72b5a1595 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:20.557605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6e65b89f-e65e-4832-803c-19f72b5a1595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-640c9082-4fc6-438d-a485-897fd1599086 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:20.560939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:20.561845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:20.562846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:20.562846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:20.562846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:20.562846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:20.562846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:20.562846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:20.562846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:20.562846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:20.567070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 322/641] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:20.567915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c9082-4fc6-438d-a485-897fd1599086 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:20.633616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 320/642] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:20.633616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65b89f-e65e-4832-803c-19f72b5a1595 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:20.687984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c9082-4fc6-438d-a485-897fd1599086 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:20.704668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:20.723608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b898345-a0df-42f8-be3b-74ee9a79e5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:20.724393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b898345-a0df-42f8-be3b-74ee9a79e5e1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.724669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b898345-a0df-42f8-be3b-74ee9a79e5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.724938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b898345-a0df-42f8-be3b-74ee9a79e5e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.726800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:20.736332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b898345-a0df-42f8-be3b-74ee9a79e5e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:20.740254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65b89f-e65e-4832-803c-19f72b5a1595 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-8b898345-a0df-42f8-be3b-74ee9a79e5e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:20.743949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:20.744924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 323/643] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:20.751495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b898345-a0df-42f8-be3b-74ee9a79e5e1 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:20.784360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a14842-d911-4ec5-bc9a-c0f5bf93dbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:20.785142 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a14842-d911-4ec5-bc9a-c0f5bf93dbda None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.785421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a14842-d911-4ec5-bc9a-c0f5bf93dbda None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.785969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a14842-d911-4ec5-bc9a-c0f5bf93dbda None admin] PATH_INFO: `/v3/users/glance` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.793231 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a14842-d911-4ec5-bc9a-c0f5bf93dbda None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:20.794715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a14842-d911-4ec5-bc9a-c0f5bf93dbda None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-51a14842-d911-4ec5-bc9a-c0f5bf93dbda None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:20.809357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:20.810340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 321/644] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:20.813632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a14842-d911-4ec5-bc9a-c0f5bf93dbda None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:20.863772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b898345-a0df-42f8-be3b-74ee9a79e5e1 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:20.901123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d59d1-99ab-4373-9c52-6b1b46c5af69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:20.901826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d59d1-99ab-4373-9c52-6b1b46c5af69 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.902087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d59d1-99ab-4373-9c52-6b1b46c5af69 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.902365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d59d1-99ab-4373-9c52-6b1b46c5af69 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.903156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d59d1-99ab-4373-9c52-6b1b46c5af69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:20.903467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d59d1-99ab-4373-9c52-6b1b46c5af69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:20.904915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d59d1-99ab-4373-9c52-6b1b46c5af69 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:20.918480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 324/645] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:20.919283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d59d1-99ab-4373-9c52-6b1b46c5af69 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:20.926782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a14842-d911-4ec5-bc9a-c0f5bf93dbda None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:20.957236 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:20.963778 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257546e-b1b5-4a7b-b48b-12a91fa3e798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:20.964575 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257546e-b1b5-4a7b-b48b-12a91fa3e798 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:20.964859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257546e-b1b5-4a7b-b48b-12a91fa3e798 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:20.965134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257546e-b1b5-4a7b-b48b-12a91fa3e798 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:20.976616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257546e-b1b5-4a7b-b48b-12a91fa3e798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-7257546e-b1b5-4a7b-b48b-12a91fa3e798 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:20.984603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:20.985511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:20.986460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 322/646] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:20.987175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257546e-b1b5-4a7b-b48b-12a91fa3e798 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:21.001310 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:21.051141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d59d1-99ab-4373-9c52-6b1b46c5af69 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:21.088652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31580a7d-f54d-4aa6-bc2c-69a15c1d56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:21.089320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580a7d-f54d-4aa6-bc2c-69a15c1d56d4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:21.089622 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580a7d-f54d-4aa6-bc2c-69a15c1d56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:21.089903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580a7d-f54d-4aa6-bc2c-69a15c1d56d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:21.090652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31580a7d-f54d-4aa6-bc2c-69a15c1d56d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:21.090953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31580a7d-f54d-4aa6-bc2c-69a15c1d56d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:21.092317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31580a7d-f54d-4aa6-bc2c-69a15c1d56d4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:21.098175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257546e-b1b5-4a7b-b48b-12a91fa3e798 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:21.101268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 325/647] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:21.102286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31580a7d-f54d-4aa6-bc2c-69a15c1d56d4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:21.133442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db055-e1a5-4a42-af10-3bffdf0ad8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:21.134149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db055-e1a5-4a42-af10-3bffdf0ad8c8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:21.134432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db055-e1a5-4a42-af10-3bffdf0ad8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:21.134707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db055-e1a5-4a42-af10-3bffdf0ad8c8 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:21.135752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db055-e1a5-4a42-af10-3bffdf0ad8c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:21.136077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db055-e1a5-4a42-af10-3bffdf0ad8c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:21.137566 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db055-e1a5-4a42-af10-3bffdf0ad8c8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:21.160010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 323/648] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 1 19:40:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:21.161183 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db055-e1a5-4a42-af10-3bffdf0ad8c8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:21.211629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31580a7d-f54d-4aa6-bc2c-69a15c1d56d4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:21.212590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:21.229765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:21.250682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0343a7f-fa86-4a71-ba05-bd518c381a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:21.251413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0343a7f-fa86-4a71-ba05-bd518c381a52 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:21.251693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0343a7f-fa86-4a71-ba05-bd518c381a52 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:21.251968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0343a7f-fa86-4a71-ba05-bd518c381a52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:21.252798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0343a7f-fa86-4a71-ba05-bd518c381a52 None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=ad06f31859f9413f9db6039ac2abf982) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:21.253107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0343a7f-fa86-4a71-ba05-bd518c381a52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:21.254614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0343a7f-fa86-4a71-ba05-bd518c381a52 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:21.265619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 326/649] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:21 2022] GET /identity/v3/role_assignments?user.id=ad06f31859f9413f9db6039ac2abf982&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:21.265619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0343a7f-fa86-4a71-ba05-bd518c381a52 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:21.280472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db055-e1a5-4a42-af10-3bffdf0ad8c8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:21.325949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad7785-2a50-4a93-90d8-6f5fc4866143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:21.326928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad7785-2a50-4a93-90d8-6f5fc4866143 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:21.327280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad7785-2a50-4a93-90d8-6f5fc4866143 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:21.327752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad7785-2a50-4a93-90d8-6f5fc4866143 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:21.329236 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad7785-2a50-4a93-90d8-6f5fc4866143 None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:21.329543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad7785-2a50-4a93-90d8-6f5fc4866143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:21.331274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad7785-2a50-4a93-90d8-6f5fc4866143 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:21.341536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 324/650] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:21.342689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad7785-2a50-4a93-90d8-6f5fc4866143 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:21.412171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0343a7f-fa86-4a71-ba05-bd518c381a52 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:21.462739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda23e86-1971-4c3f-93d7-4f411b8bfd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:21.464175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda23e86-1971-4c3f-93d7-4f411b8bfd60 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:21.464696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda23e86-1971-4c3f-93d7-4f411b8bfd60 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:21.465209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda23e86-1971-4c3f-93d7-4f411b8bfd60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:21.465535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad7785-2a50-4a93-90d8-6f5fc4866143 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:21.476918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:21.479205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda23e86-1971-4c3f-93d7-4f411b8bfd60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:21.481235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda23e86-1971-4c3f-93d7-4f411b8bfd60 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-bda23e86-1971-4c3f-93d7-4f411b8bfd60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:21.507358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:21.508704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:21.510327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:21.510327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:21.510327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:21.510327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:21.510327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 327/651] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:21.510327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda23e86-1971-4c3f-93d7-4f411b8bfd60 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:21.514423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb5017-4876-4beb-bace-26ff3c2b120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:21.515274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb5017-4876-4beb-bace-26ff3c2b120f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:21.516255 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb5017-4876-4beb-bace-26ff3c2b120f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:21.516679 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb5017-4876-4beb-bace-26ff3c2b120f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:21.561617 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:21.608323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb5017-4876-4beb-bace-26ff3c2b120f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:21.610634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb5017-4876-4beb-bace-26ff3c2b120f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:21.639142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda23e86-1971-4c3f-93d7-4f411b8bfd60 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:21.676360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af05a8f-01a9-4f9e-b927-97775a3a6169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:21.677442 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af05a8f-01a9-4f9e-b927-97775a3a6169 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:21.677927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af05a8f-01a9-4f9e-b927-97775a3a6169 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:21.678204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af05a8f-01a9-4f9e-b927-97775a3a6169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:21.689528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af05a8f-01a9-4f9e-b927-97775a3a6169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:21.691350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af05a8f-01a9-4f9e-b927-97775a3a6169 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6af05a8f-01a9-4f9e-b927-97775a3a6169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:21.710696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:21.711825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:21.712936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:21.712936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:21.712936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:21.712936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:21.719566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 328/652] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:21.719566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af05a8f-01a9-4f9e-b927-97775a3a6169 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:21.743793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 325/653] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:21.744984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb5017-4876-4beb-bace-26ff3c2b120f None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:21.773981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:21.821304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:21.841208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af05a8f-01a9-4f9e-b927-97775a3a6169 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:21.857168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb5017-4876-4beb-bace-26ff3c2b120f None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:21.879370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af5e2b-639c-4fe7-b9a7-8c836798a5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:21.880040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af5e2b-639c-4fe7-b9a7-8c836798a5f8 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:21.880751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af5e2b-639c-4fe7-b9a7-8c836798a5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:21.881036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af5e2b-639c-4fe7-b9a7-8c836798a5f8 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:21.881801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af5e2b-639c-4fe7-b9a7-8c836798a5f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:21.882109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af5e2b-639c-4fe7-b9a7-8c836798a5f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:21.883489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af5e2b-639c-4fe7-b9a7-8c836798a5f8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:21.896966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757854f-8b0f-4454-a83b-ba71dd266b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:21.897514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 329/654] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:21.897650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757854f-8b0f-4454-a83b-ba71dd266b34 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:21.897925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757854f-8b0f-4454-a83b-ba71dd266b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:21.898138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af5e2b-639c-4fe7-b9a7-8c836798a5f8 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:21.898249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757854f-8b0f-4454-a83b-ba71dd266b34 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:21.898906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757854f-8b0f-4454-a83b-ba71dd266b34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:21.899211 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757854f-8b0f-4454-a83b-ba71dd266b34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:21.900627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757854f-8b0f-4454-a83b-ba71dd266b34 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:21.918527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 326/655] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:21.920148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd62573-98e0-4c85-9291-4b7a6a6d19d9 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:21.920672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd62573-98e0-4c85-9291-4b7a6a6d19d9 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:21.921016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd62573-98e0-4c85-9291-4b7a6a6d19d9 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:21.925616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 327/656] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:21.925616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd62573-98e0-4c85-9291-4b7a6a6d19d9 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:21.996732 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:22.009736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af5e2b-639c-4fe7-b9a7-8c836798a5f8 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:22.030785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd62573-98e0-4c85-9291-4b7a6a6d19d9 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:22.046253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7595d16-a91e-42f8-939d-d3d3fd80ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:22.047249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7595d16-a91e-42f8-939d-d3d3fd80ad22 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.047608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7595d16-a91e-42f8-939d-d3d3fd80ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.047840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7595d16-a91e-42f8-939d-d3d3fd80ad22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.059762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7595d16-a91e-42f8-939d-d3d3fd80ad22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:22.061208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7595d16-a91e-42f8-939d-d3d3fd80ad22 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:22.064883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:22.069045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08416b5a-dbae-4e32-a6ed-ddc8a802b766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:22.069917 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08416b5a-dbae-4e32-a6ed-ddc8a802b766 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.070224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08416b5a-dbae-4e32-a6ed-ddc8a802b766 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.070477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08416b5a-dbae-4e32-a6ed-ddc8a802b766 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.078655 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08416b5a-dbae-4e32-a6ed-ddc8a802b766 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:22.080092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08416b5a-dbae-4e32-a6ed-ddc8a802b766 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-f7595d16-a91e-42f8-939d-d3d3fd80ad22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:22.081483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:22.082334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:22.083199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:22.083199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:22.083199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:22.083199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:22.083199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 330/657] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:22.084092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7595d16-a91e-42f8-939d-d3d3fd80ad22 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-08416b5a-dbae-4e32-a6ed-ddc8a802b766 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:22.101169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:22.102309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:22.103217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 328/658] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:22.103787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72338ecb-0e49-4041-b843-a62f12abdb00 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.103852 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72338ecb-0e49-4041-b843-a62f12abdb00 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.104497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72338ecb-0e49-4041-b843-a62f12abdb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.120726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-72338ecb-0e49-4041-b843-a62f12abdb00 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:22.121952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-72338ecb-0e49-4041-b843-a62f12abdb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:22.197632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 329/659] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:22.197850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7595d16-a91e-42f8-939d-d3d3fd80ad22 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:22.197980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72338ecb-0e49-4041-b843-a62f12abdb00 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:22.242176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8a3c3-c5e3-49a8-baa5-483c3b35df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:22.243077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8a3c3-c5e3-49a8-baa5-483c3b35df72 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.243406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8a3c3-c5e3-49a8-baa5-483c3b35df72 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.243747 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8a3c3-c5e3-49a8-baa5-483c3b35df72 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.251710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8a3c3-c5e3-49a8-baa5-483c3b35df72 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:22.253179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8a3c3-c5e3-49a8-baa5-483c3b35df72 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-8ba8a3c3-c5e3-49a8-baa5-483c3b35df72 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:22.268726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:22.269900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:22.271027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:22.271027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:22.271027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:22.271027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:22.271027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:22.271027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:22.271027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:22.271027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:22.271027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 19:40:22.271027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:22.271847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 331/660] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:22.272680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8a3c3-c5e3-49a8-baa5-483c3b35df72 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:22.274925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:22.321129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72338ecb-0e49-4041-b843-a62f12abdb00 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:22.331173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:22.356217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ba52f-5f3c-4c53-97a8-1207ac3e2dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:22.356960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba52f-5f3c-4c53-97a8-1207ac3e2dfa None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.357326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba52f-5f3c-4c53-97a8-1207ac3e2dfa None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.357694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba52f-5f3c-4c53-97a8-1207ac3e2dfa None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.358481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ba52f-5f3c-4c53-97a8-1207ac3e2dfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:22.358848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ba52f-5f3c-4c53-97a8-1207ac3e2dfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:22.360691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ba52f-5f3c-4c53-97a8-1207ac3e2dfa None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:22.372151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 330/661] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:22.373029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ba52f-5f3c-4c53-97a8-1207ac3e2dfa None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:22.388507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8a3c3-c5e3-49a8-baa5-483c3b35df72 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:22.423864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05f3e5-647d-4a7b-8e3d-46b277fbe349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:22.424530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05f3e5-647d-4a7b-8e3d-46b277fbe349 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.424808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05f3e5-647d-4a7b-8e3d-46b277fbe349 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.425074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05f3e5-647d-4a7b-8e3d-46b277fbe349 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.425933 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05f3e5-647d-4a7b-8e3d-46b277fbe349 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:22.426265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05f3e5-647d-4a7b-8e3d-46b277fbe349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:22.427697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05f3e5-647d-4a7b-8e3d-46b277fbe349 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:22.451296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 332/662] 10.222.0.66 () {56 vars in 1137 bytes} [Fri Jul 1 19:40:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:22.451296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05f3e5-647d-4a7b-8e3d-46b277fbe349 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:22.489700 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ba52f-5f3c-4c53-97a8-1207ac3e2dfa None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:22.517067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:22.533884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f566e28-41e7-45a6-b65d-4e0029dc21a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:22.534684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f566e28-41e7-45a6-b65d-4e0029dc21a0 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.534972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f566e28-41e7-45a6-b65d-4e0029dc21a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.535248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f566e28-41e7-45a6-b65d-4e0029dc21a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.546448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f566e28-41e7-45a6-b65d-4e0029dc21a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-5f566e28-41e7-45a6-b65d-4e0029dc21a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:22.555031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:22.556111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:22.556927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 331/663] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:22.557678 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f566e28-41e7-45a6-b65d-4e0029dc21a0 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:22.567121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05f3e5-647d-4a7b-8e3d-46b277fbe349 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:22.583758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:22.605598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43646f5c-1b8b-4818-9e03-d717ae0d89ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:22.606270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43646f5c-1b8b-4818-9e03-d717ae0d89ff None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.606553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43646f5c-1b8b-4818-9e03-d717ae0d89ff None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.606823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43646f5c-1b8b-4818-9e03-d717ae0d89ff None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.607659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43646f5c-1b8b-4818-9e03-d717ae0d89ff None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:22.607967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43646f5c-1b8b-4818-9e03-d717ae0d89ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:22.609926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43646f5c-1b8b-4818-9e03-d717ae0d89ff None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:22.628707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 333/664] 10.222.0.66 () {56 vars in 1139 bytes} [Fri Jul 1 19:40:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:22.633768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43646f5c-1b8b-4818-9e03-d717ae0d89ff None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:22.678298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f566e28-41e7-45a6-b65d-4e0029dc21a0 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:22.726159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bbb1a-d19c-4749-af9a-495e88612221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:22.727046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bbb1a-d19c-4749-af9a-495e88612221 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.727381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bbb1a-d19c-4749-af9a-495e88612221 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.727717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bbb1a-d19c-4749-af9a-495e88612221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.735010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bbb1a-d19c-4749-af9a-495e88612221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:22.736529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bbb1a-d19c-4749-af9a-495e88612221 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:22.737157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43646f5c-1b8b-4818-9e03-d717ae0d89ff None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-0d5bbb1a-d19c-4749-af9a-495e88612221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:22.751411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:22.752320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:22.753229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:22.753229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:22.753229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:22.753229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:22.753229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:22.753229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:22.753229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:22.753229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:22.753229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 332/665] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:22.753896 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bbb1a-d19c-4749-af9a-495e88612221 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:22.778807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393449e-73d9-4964-8a96-dec41ac61885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:22.779390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393449e-73d9-4964-8a96-dec41ac61885 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.779661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393449e-73d9-4964-8a96-dec41ac61885 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.779934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393449e-73d9-4964-8a96-dec41ac61885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.802120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:22.826384 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:22.855913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393449e-73d9-4964-8a96-dec41ac61885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:22.857368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393449e-73d9-4964-8a96-dec41ac61885 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:22.865822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bbb1a-d19c-4749-af9a-495e88612221 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:22.903346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32df48-a677-4841-9118-60a2c2abdfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:22.904039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32df48-a677-4841-9118-60a2c2abdfb8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:22.904318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32df48-a677-4841-9118-60a2c2abdfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:22.904593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32df48-a677-4841-9118-60a2c2abdfb8 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:22.905333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32df48-a677-4841-9118-60a2c2abdfb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:22.905672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32df48-a677-4841-9118-60a2c2abdfb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:22.907093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32df48-a677-4841-9118-60a2c2abdfb8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:22.919975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 333/666] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:22.919975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32df48-a677-4841-9118-60a2c2abdfb8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:22.964378 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 334/667] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:22.965477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393449e-73d9-4964-8a96-dec41ac61885 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:23.032915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32df48-a677-4841-9118-60a2c2abdfb8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:23.034857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:23.042121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:23.073550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233be8f1-797f-4075-b919-da2235bf7b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:23.074525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233be8f1-797f-4075-b919-da2235bf7b29 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.074955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233be8f1-797f-4075-b919-da2235bf7b29 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.075377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233be8f1-797f-4075-b919-da2235bf7b29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.078702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393449e-73d9-4964-8a96-dec41ac61885 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:23.117074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd2dfc-5686-43d1-ad12-9b4e5498cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:23.117763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd2dfc-5686-43d1-ad12-9b4e5498cb85 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.118040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd2dfc-5686-43d1-ad12-9b4e5498cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.118311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd2dfc-5686-43d1-ad12-9b4e5498cb85 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.119092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd2dfc-5686-43d1-ad12-9b4e5498cb85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:23.119392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd2dfc-5686-43d1-ad12-9b4e5498cb85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:23.120819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd2dfc-5686-43d1-ad12-9b4e5498cb85 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:23.133357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 335/668] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:23.134187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd2dfc-5686-43d1-ad12-9b4e5498cb85 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:23.159013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233be8f1-797f-4075-b919-da2235bf7b29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:23.160477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233be8f1-797f-4075-b919-da2235bf7b29 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:23.246425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd2dfc-5686-43d1-ad12-9b4e5498cb85 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:23.269097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 334/669] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:23.270382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233be8f1-797f-4075-b919-da2235bf7b29 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:23.275357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:23.284165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:23.290380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3773eb-4387-43f1-97e4-3064855316b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:23.291172 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3773eb-4387-43f1-97e4-3064855316b1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.291447 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3773eb-4387-43f1-97e4-3064855316b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.291717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3773eb-4387-43f1-97e4-3064855316b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.303659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3773eb-4387-43f1-97e4-3064855316b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:23.305129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3773eb-4387-43f1-97e4-3064855316b1 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-5b3773eb-4387-43f1-97e4-3064855316b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:23.322907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:23.323855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:23.324739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:23.324739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:23.324739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:23.324739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:23.324739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 336/670] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:23.325359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3773eb-4387-43f1-97e4-3064855316b1 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:23.395882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233be8f1-797f-4075-b919-da2235bf7b29 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:23.433603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9adee9-d458-4fe4-a00f-f7e25dba2d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:23.434535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9adee9-d458-4fe4-a00f-f7e25dba2d58 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.434821 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9adee9-d458-4fe4-a00f-f7e25dba2d58 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.435095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9adee9-d458-4fe4-a00f-f7e25dba2d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.446397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3773eb-4387-43f1-97e4-3064855316b1 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:23.446553 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9adee9-d458-4fe4-a00f-f7e25dba2d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:23.448146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9adee9-d458-4fe4-a00f-f7e25dba2d58 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-be9adee9-d458-4fe4-a00f-f7e25dba2d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:23.468120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:23.469706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:23.470587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:23.470587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:23.470587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:23.470587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:23.470587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 335/671] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:23.471418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9adee9-d458-4fe4-a00f-f7e25dba2d58 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:23.487611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddeae5-223d-4f7f-9e94-8dc36124ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:23.488274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddeae5-223d-4f7f-9e94-8dc36124ede9 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.488560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddeae5-223d-4f7f-9e94-8dc36124ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.488837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddeae5-223d-4f7f-9e94-8dc36124ede9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.489727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddeae5-223d-4f7f-9e94-8dc36124ede9 None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=c022224b03a54ab489d95502c63a25dd) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:23.490037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddeae5-223d-4f7f-9e94-8dc36124ede9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:23.491442 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddeae5-223d-4f7f-9e94-8dc36124ede9 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:23.497656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 337/672] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:23 2022] GET /identity/v3/role_assignments?user.id=c022224b03a54ab489d95502c63a25dd&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:23.498446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddeae5-223d-4f7f-9e94-8dc36124ede9 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:23.554735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:23.572337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:23.608074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9adee9-d458-4fe4-a00f-f7e25dba2d58 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:23.632203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddeae5-223d-4f7f-9e94-8dc36124ede9 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:23.647241 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace58b2-6156-43d3-b223-e2b171058859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:23.648131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace58b2-6156-43d3-b223-e2b171058859 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.649103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace58b2-6156-43d3-b223-e2b171058859 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.649103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace58b2-6156-43d3-b223-e2b171058859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.663600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace58b2-6156-43d3-b223-e2b171058859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:23.665369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace58b2-6156-43d3-b223-e2b171058859 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:23.672851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f494fc6-247f-4c8c-b240-1606effd2566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:23.673677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494fc6-247f-4c8c-b240-1606effd2566 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.673809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494fc6-247f-4c8c-b240-1606effd2566 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.674084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494fc6-247f-4c8c-b240-1606effd2566 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.674818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f494fc6-247f-4c8c-b240-1606effd2566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:23.675124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f494fc6-247f-4c8c-b240-1606effd2566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:23.676504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f494fc6-247f-4c8c-b240-1606effd2566 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-5ace58b2-6156-43d3-b223-e2b171058859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:23.687070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:23.689049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:23.690068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:23.690068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:23.690068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:23.690068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:23.690068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 336/673] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:23.690336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 338/674] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:23.690415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace58b2-6156-43d3-b223-e2b171058859 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:23.690973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d3736-5446-4cac-b311-044069c5307e None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.692619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d3736-5446-4cac-b311-044069c5307e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.692619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d3736-5446-4cac-b311-044069c5307e None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.694471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 339/675] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:23.695461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d3736-5446-4cac-b311-044069c5307e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:23.801283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace58b2-6156-43d3-b223-e2b171058859 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:23.810102 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:23.814206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d3736-5446-4cac-b311-044069c5307e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:23.824112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:23.842399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db032282-1714-451d-9ddf-a0d0acc66c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:23.843060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db032282-1714-451d-9ddf-a0d0acc66c9f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.843341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db032282-1714-451d-9ddf-a0d0acc66c9f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.843615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db032282-1714-451d-9ddf-a0d0acc66c9f None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.844335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db032282-1714-451d-9ddf-a0d0acc66c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:23.844640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db032282-1714-451d-9ddf-a0d0acc66c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:23.846122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db032282-1714-451d-9ddf-a0d0acc66c9f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:23.850815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990194f2-0711-4819-b362-0e59a7530242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:23.851601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990194f2-0711-4819-b362-0e59a7530242 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.851882 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990194f2-0711-4819-b362-0e59a7530242 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.852155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990194f2-0711-4819-b362-0e59a7530242 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.862370 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 337/676] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:23.862370 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db032282-1714-451d-9ddf-a0d0acc66c9f None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:23.864656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990194f2-0711-4819-b362-0e59a7530242 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:23.865747 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990194f2-0711-4819-b362-0e59a7530242 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-990194f2-0711-4819-b362-0e59a7530242 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:23.881456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:23.882507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:23.883433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:23.883433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:23.883433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:23.883433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:23.883433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:23.883433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:23.883433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:23.883433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:23.883433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:23.883433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:23.887481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 340/677] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:23.888643 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81301c-56e8-4390-b0ae-9cd331311c41 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:23.888932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81301c-56e8-4390-b0ae-9cd331311c41 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:23.889207 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81301c-56e8-4390-b0ae-9cd331311c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:23.908600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-4d81301c-56e8-4390-b0ae-9cd331311c41 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:23.909823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4d81301c-56e8-4390-b0ae-9cd331311c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:23.972921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db032282-1714-451d-9ddf-a0d0acc66c9f None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:23.981624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 341/678] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:23.981624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81301c-56e8-4390-b0ae-9cd331311c41 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.017756 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee41fff-59b5-4e0b-87bb-74ddc55d17b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:24.018427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee41fff-59b5-4e0b-87bb-74ddc55d17b7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:24.018715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee41fff-59b5-4e0b-87bb-74ddc55d17b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:24.018991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee41fff-59b5-4e0b-87bb-74ddc55d17b7 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:24.019711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee41fff-59b5-4e0b-87bb-74ddc55d17b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:24.020015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee41fff-59b5-4e0b-87bb-74ddc55d17b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:24.021403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee41fff-59b5-4e0b-87bb-74ddc55d17b7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:24.033946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 338/679] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:24.034748 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee41fff-59b5-4e0b-87bb-74ddc55d17b7 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.064464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:24.087726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:24.091663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81301c-56e8-4390-b0ae-9cd331311c41 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:24.129325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7eabe-c478-47f2-9e33-00bace9c2005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:24.130171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7eabe-c478-47f2-9e33-00bace9c2005 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:24.130448 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7eabe-c478-47f2-9e33-00bace9c2005 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:24.130726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7eabe-c478-47f2-9e33-00bace9c2005 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:24.137573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7eabe-c478-47f2-9e33-00bace9c2005 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:24.139068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7eabe-c478-47f2-9e33-00bace9c2005 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:24.142674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee41fff-59b5-4e0b-87bb-74ddc55d17b7 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-45f7eabe-c478-47f2-9e33-00bace9c2005 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:24.154971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:24.155837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:24.156791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:24.156791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:24.156791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:24.156791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:24.156791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:24.156791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:24.156791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:24.156791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:24.156791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 342/680] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:24.157266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7eabe-c478-47f2-9e33-00bace9c2005 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.184217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8722e0d-b885-41a7-aa48-01be7a307a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:24.185085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8722e0d-b885-41a7-aa48-01be7a307a40 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:24.185469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8722e0d-b885-41a7-aa48-01be7a307a40 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:24.185782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8722e0d-b885-41a7-aa48-01be7a307a40 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:24.186612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8722e0d-b885-41a7-aa48-01be7a307a40 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:24.186918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8722e0d-b885-41a7-aa48-01be7a307a40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:24.188966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8722e0d-b885-41a7-aa48-01be7a307a40 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:24.207962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 339/681] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 1 19:40:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:24.208788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8722e0d-b885-41a7-aa48-01be7a307a40 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.273768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7eabe-c478-47f2-9e33-00bace9c2005 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:24.298244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:24.309449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a354c2-2b5f-40ea-8e86-c6d41de6b542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:24.310597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a354c2-2b5f-40ea-8e86-c6d41de6b542 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:24.311153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a354c2-2b5f-40ea-8e86-c6d41de6b542 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:24.311436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a354c2-2b5f-40ea-8e86-c6d41de6b542 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:24.316461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8722e0d-b885-41a7-aa48-01be7a307a40 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:24.324544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a354c2-2b5f-40ea-8e86-c6d41de6b542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-23a354c2-2b5f-40ea-8e86-c6d41de6b542 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:24.335215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:24.336234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 343/682] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:24.342542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a354c2-2b5f-40ea-8e86-c6d41de6b542 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.352251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce361791-5768-4eba-86b9-10fadfce0f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:24.353545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce361791-5768-4eba-86b9-10fadfce0f4c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:24.353972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce361791-5768-4eba-86b9-10fadfce0f4c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:24.354695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce361791-5768-4eba-86b9-10fadfce0f4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:24.362084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce361791-5768-4eba-86b9-10fadfce0f4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:24.363481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:24.363738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce361791-5768-4eba-86b9-10fadfce0f4c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-ce361791-5768-4eba-86b9-10fadfce0f4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:24.379252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:24.380307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:24.381208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:24.381208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:24.381208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:24.381208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:24.381208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:24.381208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:24.381208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:24.381208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:24.381208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 340/683] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:24.381740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce361791-5768-4eba-86b9-10fadfce0f4c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.456137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a354c2-2b5f-40ea-8e86-c6d41de6b542 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:24.490640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53e5b6-4253-4960-b0c6-6a4a651e2024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:24.491286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e5b6-4253-4960-b0c6-6a4a651e2024 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:24.491564 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e5b6-4253-4960-b0c6-6a4a651e2024 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:24.491832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53e5b6-4253-4960-b0c6-6a4a651e2024 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:24.492615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53e5b6-4253-4960-b0c6-6a4a651e2024 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:24.492913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53e5b6-4253-4960-b0c6-6a4a651e2024 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:24.494367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53e5b6-4253-4960-b0c6-6a4a651e2024 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:24.496123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce361791-5768-4eba-86b9-10fadfce0f4c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:24.507762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 344/684] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:24.508585 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53e5b6-4253-4960-b0c6-6a4a651e2024 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.540893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b9a60-cdf8-4f88-99d8-3ef7820e538d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:24.541531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b9a60-cdf8-4f88-99d8-3ef7820e538d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:24.541838 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b9a60-cdf8-4f88-99d8-3ef7820e538d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:24.542254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b9a60-cdf8-4f88-99d8-3ef7820e538d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:24.563406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:24.598155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:24.623499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b9a60-cdf8-4f88-99d8-3ef7820e538d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:24.623790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53e5b6-4253-4960-b0c6-6a4a651e2024 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:24.624986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b9a60-cdf8-4f88-99d8-3ef7820e538d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:24.661802 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef188d31-edda-4545-80aa-e52730f769a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:24.662103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef188d31-edda-4545-80aa-e52730f769a8 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:24.662381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef188d31-edda-4545-80aa-e52730f769a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:24.662646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef188d31-edda-4545-80aa-e52730f769a8 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:24.663388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef188d31-edda-4545-80aa-e52730f769a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:24.663686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef188d31-edda-4545-80aa-e52730f769a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:24.665083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef188d31-edda-4545-80aa-e52730f769a8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:24.674825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 345/685] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:24.675626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef188d31-edda-4545-80aa-e52730f769a8 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.738465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 341/686] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:24.739319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b9a60-cdf8-4f88-99d8-3ef7820e538d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.785966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:24.790595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef188d31-edda-4545-80aa-e52730f769a8 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:24.829554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d9e0-3f88-4ad1-939e-1abdb2df8ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:24.830370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d9e0-3f88-4ad1-939e-1abdb2df8ef7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:24.830855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d9e0-3f88-4ad1-939e-1abdb2df8ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:24.831138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d9e0-3f88-4ad1-939e-1abdb2df8ef7 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:24.832033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621d9e0-3f88-4ad1-939e-1abdb2df8ef7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:24.832431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621d9e0-3f88-4ad1-939e-1abdb2df8ef7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:24.834002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621d9e0-3f88-4ad1-939e-1abdb2df8ef7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:24.841445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b9a60-cdf8-4f88-99d8-3ef7820e538d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:24.844890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:24.848946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 346/687] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:24.850033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d9e0-3f88-4ad1-939e-1abdb2df8ef7 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.881787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909861b-1a0a-42ea-a299-4ffe5d4757e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:24.882462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909861b-1a0a-42ea-a299-4ffe5d4757e1 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:24.882742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909861b-1a0a-42ea-a299-4ffe5d4757e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:24.883012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909861b-1a0a-42ea-a299-4ffe5d4757e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:24.883833 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909861b-1a0a-42ea-a299-4ffe5d4757e1 None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=a4b0e6ea57af454cbc8f5bd180bc052e) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:24.884139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909861b-1a0a-42ea-a299-4ffe5d4757e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:24.885562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909861b-1a0a-42ea-a299-4ffe5d4757e1 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:24.892383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 342/688] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:24 2022] GET /identity/v3/role_assignments?user.id=a4b0e6ea57af454cbc8f5bd180bc052e&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:24.893155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909861b-1a0a-42ea-a299-4ffe5d4757e1 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:24.979227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d9e0-3f88-4ad1-939e-1abdb2df8ef7 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:25.029732 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909861b-1a0a-42ea-a299-4ffe5d4757e1 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:25.031270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a98458-c4f2-47e3-8438-72c6c45a1a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:25.032465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a98458-c4f2-47e3-8438-72c6c45a1a13 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.032822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a98458-c4f2-47e3-8438-72c6c45a1a13 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.033169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a98458-c4f2-47e3-8438-72c6c45a1a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.042008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:25.051178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a98458-c4f2-47e3-8438-72c6c45a1a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:25.052911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a98458-c4f2-47e3-8438-72c6c45a1a13 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:25.074610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c408927-8b53-4fd5-b091-8764dec6fede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-c3a98458-c4f2-47e3-8438-72c6c45a1a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:25.074815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:25.075655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:25.076551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:25.076551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:25.076551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:25.076551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:25.076551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 347/689] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:25.077023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a98458-c4f2-47e3-8438-72c6c45a1a13 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:25.078671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c408927-8b53-4fd5-b091-8764dec6fede None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.079885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c408927-8b53-4fd5-b091-8764dec6fede None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.080332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c408927-8b53-4fd5-b091-8764dec6fede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.097668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c408927-8b53-4fd5-b091-8764dec6fede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:25.099449 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c408927-8b53-4fd5-b091-8764dec6fede None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-0c408927-8b53-4fd5-b091-8764dec6fede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:25.120188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:25.121145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:25.122067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:25.122067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:25.122067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:25.122067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:25.122067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 343/690] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:25.122517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c408927-8b53-4fd5-b091-8764dec6fede None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:25.147996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:25.204932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a98458-c4f2-47e3-8438-72c6c45a1a13 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:25.240654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738b090-b955-46bf-a584-effa9110ca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:25.241313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738b090-b955-46bf-a584-effa9110ca47 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.241605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738b090-b955-46bf-a584-effa9110ca47 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.241909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738b090-b955-46bf-a584-effa9110ca47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.242744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738b090-b955-46bf-a584-effa9110ca47 None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=370f1c0f7af04a56bdf8fa452550380a) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:25.243053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738b090-b955-46bf-a584-effa9110ca47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:25.244483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738b090-b955-46bf-a584-effa9110ca47 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:25.251552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 348/691] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:25 2022] GET /identity/v3/role_assignments?user.id=370f1c0f7af04a56bdf8fa452550380a&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:25.257708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738b090-b955-46bf-a584-effa9110ca47 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:25.286596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c408927-8b53-4fd5-b091-8764dec6fede None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:25.353635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f70141-c6fa-4338-b84d-6f6b0e48435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:25.354549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f70141-c6fa-4338-b84d-6f6b0e48435a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.355494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f70141-c6fa-4338-b84d-6f6b0e48435a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.355919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f70141-c6fa-4338-b84d-6f6b0e48435a None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.357734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f70141-c6fa-4338-b84d-6f6b0e48435a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:25.361901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f70141-c6fa-4338-b84d-6f6b0e48435a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:25.363534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f70141-c6fa-4338-b84d-6f6b0e48435a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:25.383601 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 344/692] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:25.389155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f70141-c6fa-4338-b84d-6f6b0e48435a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:25.436581 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:25.458936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738b090-b955-46bf-a584-effa9110ca47 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:25.490657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:25.518113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058f090-b6d6-43e8-a1d4-2ecffa20270b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:25.519082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f090-b6d6-43e8-a1d4-2ecffa20270b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.519549 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f090-b6d6-43e8-a1d4-2ecffa20270b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.520025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f090-b6d6-43e8-a1d4-2ecffa20270b None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.520930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058f090-b6d6-43e8-a1d4-2ecffa20270b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:25.521411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058f090-b6d6-43e8-a1d4-2ecffa20270b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:25.523049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058f090-b6d6-43e8-a1d4-2ecffa20270b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:25.538420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 349/693] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:25.544087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058f090-b6d6-43e8-a1d4-2ecffa20270b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:25.560297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f70141-c6fa-4338-b84d-6f6b0e48435a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:25.608276 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8e313-3d7f-4696-80aa-10e3793e74de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:25.609637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8e313-3d7f-4696-80aa-10e3793e74de None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.610564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8e313-3d7f-4696-80aa-10e3793e74de None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.610905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8e313-3d7f-4696-80aa-10e3793e74de None admin] PATH_INFO: `/v3/users/nova` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.621363 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8e313-3d7f-4696-80aa-10e3793e74de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:25.623587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8e313-3d7f-4696-80aa-10e3793e74de None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-62f8e313-3d7f-4696-80aa-10e3793e74de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:25.647496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:25.648481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:25.649431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 345/694] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 1 19:40:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:25.654456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8e313-3d7f-4696-80aa-10e3793e74de None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:25.676814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058f090-b6d6-43e8-a1d4-2ecffa20270b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:25.727275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2b2be-1c3a-442d-9fbc-8363def37783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:25.728458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2b2be-1c3a-442d-9fbc-8363def37783 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.729155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2b2be-1c3a-442d-9fbc-8363def37783 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.729659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2b2be-1c3a-442d-9fbc-8363def37783 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.739970 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2b2be-1c3a-442d-9fbc-8363def37783 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:25.742394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e2b2be-1c3a-442d-9fbc-8363def37783 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:25.757248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-e2e2b2be-1c3a-442d-9fbc-8363def37783 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:25.768065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:25.769148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:25.770269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:25.770269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:25.770269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:25.770269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:25.770269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:25.770269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:25.770269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:25.770269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:25.773013 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 350/695] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:25.775182 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c12d3-8801-4bde-b2e0-994a55cc0f22 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.775864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c12d3-8801-4bde-b2e0-994a55cc0f22 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.776309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c12d3-8801-4bde-b2e0-994a55cc0f22 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.778719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 351/696] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:25.783817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c12d3-8801-4bde-b2e0-994a55cc0f22 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:25.790925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8e313-3d7f-4696-80aa-10e3793e74de None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:25.835904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b03e75-aaf2-4a67-b81b-e916f735ef3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:25.836925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03e75-aaf2-4a67-b81b-e916f735ef3c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.837435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03e75-aaf2-4a67-b81b-e916f735ef3c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.838056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03e75-aaf2-4a67-b81b-e916f735ef3c None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.838834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b03e75-aaf2-4a67-b81b-e916f735ef3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:25.839146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b03e75-aaf2-4a67-b81b-e916f735ef3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:25.840587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b03e75-aaf2-4a67-b81b-e916f735ef3c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:25.841988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:25.864311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 346/697] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 1 19:40:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:25.866152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba8eae-f17b-4188-abad-043c169ec2ff None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.866152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba8eae-f17b-4188-abad-043c169ec2ff None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.866355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba8eae-f17b-4188-abad-043c169ec2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.883634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-59ba8eae-f17b-4188-abad-043c169ec2ff None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:25.884845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-59ba8eae-f17b-4188-abad-043c169ec2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:25.906425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c12d3-8801-4bde-b2e0-994a55cc0f22 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:25.954021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bd0a4-c7a5-40d4-85d2-315dc30932f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:25.955035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bd0a4-c7a5-40d4-85d2-315dc30932f5 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:25.955524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bd0a4-c7a5-40d4-85d2-315dc30932f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:25.956025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bd0a4-c7a5-40d4-85d2-315dc30932f5 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:25.957020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bd0a4-c7a5-40d4-85d2-315dc30932f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:25.957533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bd0a4-c7a5-40d4-85d2-315dc30932f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:25.959236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bd0a4-c7a5-40d4-85d2-315dc30932f5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:25.961914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 347/698] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:25.963010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba8eae-f17b-4188-abad-043c169ec2ff None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:25.975548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 352/699] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:25.982174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bd0a4-c7a5-40d4-85d2-315dc30932f5 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:26.054246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:26.088695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba8eae-f17b-4188-abad-043c169ec2ff None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:26.090715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:26.098018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bd0a4-c7a5-40d4-85d2-315dc30932f5 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:26.125628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979d8cd-0f3e-41af-8af5-fbd049c61601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:26.126209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979d8cd-0f3e-41af-8af5-fbd049c61601 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:26.126490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979d8cd-0f3e-41af-8af5-fbd049c61601 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:26.126764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979d8cd-0f3e-41af-8af5-fbd049c61601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:26.134862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5c413-dcfe-4440-9882-b1402f2ee800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:26.135704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5c413-dcfe-4440-9882-b1402f2ee800 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:26.136058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5c413-dcfe-4440-9882-b1402f2ee800 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:26.136398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5c413-dcfe-4440-9882-b1402f2ee800 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:26.137326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5c413-dcfe-4440-9882-b1402f2ee800 None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=c2bce9daec8c446d89eb6cc5950d6649) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:26.137842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5c413-dcfe-4440-9882-b1402f2ee800 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:26.139568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb5c413-dcfe-4440-9882-b1402f2ee800 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:26.153971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 353/700] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:25 2022] GET /identity/v3/role_assignments?user.id=c2bce9daec8c446d89eb6cc5950d6649&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:26.155183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5c413-dcfe-4440-9882-b1402f2ee800 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:26.201864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979d8cd-0f3e-41af-8af5-fbd049c61601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:26.203984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979d8cd-0f3e-41af-8af5-fbd049c61601 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:26.277991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:26.313273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 348/701] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:26.320291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979d8cd-0f3e-41af-8af5-fbd049c61601 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:26.375258 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5c413-dcfe-4440-9882-b1402f2ee800 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:26.426714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0b377-0ec3-40fc-be44-3c9ab391cc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:26.427524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b377-0ec3-40fc-be44-3c9ab391cc51 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:26.427804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b377-0ec3-40fc-be44-3c9ab391cc51 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:26.428174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0b377-0ec3-40fc-be44-3c9ab391cc51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:26.441613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0b377-0ec3-40fc-be44-3c9ab391cc51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:26.443196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0b377-0ec3-40fc-be44-3c9ab391cc51 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:26.459318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979d8cd-0f3e-41af-8af5-fbd049c61601 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-52a0b377-0ec3-40fc-be44-3c9ab391cc51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:26.463804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:26.464664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:26.465653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:26.465653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:26.465653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:26.465653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:26.465653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 354/702] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:26.472254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0b377-0ec3-40fc-be44-3c9ab391cc51 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:26.489673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:26.498108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c803487-981e-4ad9-8e2a-494465a028c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:26.498943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c803487-981e-4ad9-8e2a-494465a028c0 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:26.499218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c803487-981e-4ad9-8e2a-494465a028c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:26.500028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c803487-981e-4ad9-8e2a-494465a028c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:26.515077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c803487-981e-4ad9-8e2a-494465a028c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:26.517196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c803487-981e-4ad9-8e2a-494465a028c0 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-1c803487-981e-4ad9-8e2a-494465a028c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:26.536573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:26.537509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:26.538467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:26.538467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:26.538467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:26.538467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:26.538467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 349/703] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:26.543488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c803487-981e-4ad9-8e2a-494465a028c0 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:26.567540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:26.608000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0b377-0ec3-40fc-be44-3c9ab391cc51 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:26.646498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943a293-b5ba-4017-883a-a649794cc3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:26.647507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943a293-b5ba-4017-883a-a649794cc3dd None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:26.648092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943a293-b5ba-4017-883a-a649794cc3dd None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:26.648664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943a293-b5ba-4017-883a-a649794cc3dd None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:26.649651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943a293-b5ba-4017-883a-a649794cc3dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:26.650157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943a293-b5ba-4017-883a-a649794cc3dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:26.651473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c803487-981e-4ad9-8e2a-494465a028c0 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:26.651956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9943a293-b5ba-4017-883a-a649794cc3dd None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:26.666627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 355/704] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:26.673202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943a293-b5ba-4017-883a-a649794cc3dd None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:26.690290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e4ef4-300e-4997-9793-022d40ec2794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:26.691092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e4ef4-300e-4997-9793-022d40ec2794 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:26.691205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e4ef4-300e-4997-9793-022d40ec2794 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:26.691468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e4ef4-300e-4997-9793-022d40ec2794 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:26.692254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e4ef4-300e-4997-9793-022d40ec2794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:26.692630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e4ef4-300e-4997-9793-022d40ec2794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:26.694007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e4ef4-300e-4997-9793-022d40ec2794 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:26.707106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 350/705] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:26.712550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e4ef4-300e-4997-9793-022d40ec2794 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:26.755750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:26.795440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943a293-b5ba-4017-883a-a649794cc3dd None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:26.837942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:26.873237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb07202-ddfa-405c-8c8a-a1c4d099a688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:26.874083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb07202-ddfa-405c-8c8a-a1c4d099a688 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:26.874371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb07202-ddfa-405c-8c8a-a1c4d099a688 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:26.875194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb07202-ddfa-405c-8c8a-a1c4d099a688 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:26.875910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb07202-ddfa-405c-8c8a-a1c4d099a688 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:26.876565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e4ef4-300e-4997-9793-022d40ec2794 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:26.877376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb07202-ddfa-405c-8c8a-a1c4d099a688 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:26.901161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 356/706] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:40:26 2022] POST /identity/v3/users => generated 254 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:40:26.930797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc168cc-a616-4c28-9c27-1f0b56bbd6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:26.931696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc168cc-a616-4c28-9c27-1f0b56bbd6ed None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:26.932045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc168cc-a616-4c28-9c27-1f0b56bbd6ed None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:26.932385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc168cc-a616-4c28-9c27-1f0b56bbd6ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:26.940389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc168cc-a616-4c28-9c27-1f0b56bbd6ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:26.941932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc168cc-a616-4c28-9c27-1f0b56bbd6ed None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-5cc168cc-a616-4c28-9c27-1f0b56bbd6ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:26.958047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:26.959148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:26.960282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:26.960282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:26.960282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:26.960282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:26.960282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:26.960282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:26.960282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:26.960282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:26.965619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 351/707] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:26.967021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb07202-ddfa-405c-8c8a-a1c4d099a688 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:27.141249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb07202-ddfa-405c-8c8a-a1c4d099a688 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:27.169946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7dabf-49e9-426e-a500-490fb8e2e78d None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:27.170298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7dabf-49e9-426e-a500-490fb8e2e78d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:27.170482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7dabf-49e9-426e-a500-490fb8e2e78d None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:27.172207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 352/708] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:27.174920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:27.177504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8b7de-013c-41e4-8268-e137803d02a8 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:27.177823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8b7de-013c-41e4-8268-e137803d02a8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:27.178090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8b7de-013c-41e4-8268-e137803d02a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:27.199619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f4a8b7de-013c-41e4-8268-e137803d02a8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:27.200872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e630fbb-23c1-4bd6-be04-944e349b9c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:27.201007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f4a8b7de-013c-41e4-8268-e137803d02a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:27.201973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e630fbb-23c1-4bd6-be04-944e349b9c5f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:27.202468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e630fbb-23c1-4bd6-be04-944e349b9c5f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:27.203570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e630fbb-23c1-4bd6-be04-944e349b9c5f None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:27.204709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e630fbb-23c1-4bd6-be04-944e349b9c5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:27.205020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e630fbb-23c1-4bd6-be04-944e349b9c5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:27.208209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e630fbb-23c1-4bd6-be04-944e349b9c5f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:27.222097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 357/709] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:27.228317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e630fbb-23c1-4bd6-be04-944e349b9c5f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:27.314973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 353/710] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:27.339179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8b7de-013c-41e4-8268-e137803d02a8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:27.358583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e630fbb-23c1-4bd6-be04-944e349b9c5f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:27.396808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916d130-8dae-44db-bc7a-008c305a9da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:27.398625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916d130-8dae-44db-bc7a-008c305a9da0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:27.399065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916d130-8dae-44db-bc7a-008c305a9da0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:27.399585 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916d130-8dae-44db-bc7a-008c305a9da0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:27.400611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916d130-8dae-44db-bc7a-008c305a9da0 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=ad06f31859f9413f9db6039ac2abf982) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:27.401087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916d130-8dae-44db-bc7a-008c305a9da0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:27.402721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916d130-8dae-44db-bc7a-008c305a9da0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:27.409404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 358/711] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:27 2022] GET /identity/v3/role_assignments?user.id=ad06f31859f9413f9db6039ac2abf982&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:27.420092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:27.458619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8b7de-013c-41e4-8268-e137803d02a8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:27.511552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28faf6-80e8-41cf-bb53-58104a7d11eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:27.512396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28faf6-80e8-41cf-bb53-58104a7d11eb None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:27.512680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28faf6-80e8-41cf-bb53-58104a7d11eb None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:27.512956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28faf6-80e8-41cf-bb53-58104a7d11eb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:27.513722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28faf6-80e8-41cf-bb53-58104a7d11eb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:27.515523 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28faf6-80e8-41cf-bb53-58104a7d11eb None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-cc28faf6-80e8-41cf-bb53-58104a7d11eb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:27.536616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:27.537731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 19:40:27.538691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:27.549973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 354/712] 10.222.0.66 () {56 vars in 1129 bytes} [Fri Jul 1 19:40:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:27.556203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916d130-8dae-44db-bc7a-008c305a9da0 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:27.633707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:27.697020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916d130-8dae-44db-bc7a-008c305a9da0 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:27.738957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66633d0-e64d-4a6e-ab98-f067732ad16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:27.739551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66633d0-e64d-4a6e-ab98-f067732ad16a None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:27.739857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66633d0-e64d-4a6e-ab98-f067732ad16a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:27.740103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66633d0-e64d-4a6e-ab98-f067732ad16a None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:27.740813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66633d0-e64d-4a6e-ab98-f067732ad16a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:27.741121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66633d0-e64d-4a6e-ab98-f067732ad16a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:27.742520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66633d0-e64d-4a6e-ab98-f067732ad16a None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:27.751651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 359/713] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 1 19:40:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:27.757228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28faf6-80e8-41cf-bb53-58104a7d11eb None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:27.885325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28faf6-80e8-41cf-bb53-58104a7d11eb None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:27.917093 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3568d2a-8299-4740-8d03-6f938ef87abe None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:27.917629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3568d2a-8299-4740-8d03-6f938ef87abe None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:27.917629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3568d2a-8299-4740-8d03-6f938ef87abe None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:27.919145 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 360/714] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:27.924651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435f3e8-ee49-4377-8849-30b02db1a312 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:27.924947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435f3e8-ee49-4377-8849-30b02db1a312 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:27.925226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435f3e8-ee49-4377-8849-30b02db1a312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:27.941097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ee5c5-e4ff-43f3-9426-dfc8d74a5f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:27.941809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ee5c5-e4ff-43f3-9426-dfc8d74a5f5d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:27.942091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ee5c5-e4ff-43f3-9426-dfc8d74a5f5d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:27.942365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ee5c5-e4ff-43f3-9426-dfc8d74a5f5d None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:27.943083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ee5c5-e4ff-43f3-9426-dfc8d74a5f5d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:27.943388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ee5c5-e4ff-43f3-9426-dfc8d74a5f5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:27.944775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ee5c5-e4ff-43f3-9426-dfc8d74a5f5d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:27.946689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:27.953305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a435f3e8-ee49-4377-8849-30b02db1a312 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:27.954748 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a435f3e8-ee49-4377-8849-30b02db1a312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:27.955952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 355/715] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 1 19:40:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:28.090053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 361/716] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:28.128156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ee5c5-e4ff-43f3-9426-dfc8d74a5f5d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:28.241485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ee5c5-e4ff-43f3-9426-dfc8d74a5f5d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:28.285114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd373-e294-49a4-8ed9-9c6375d3307a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:28.286199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd373-e294-49a4-8ed9-9c6375d3307a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:28.286482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd373-e294-49a4-8ed9-9c6375d3307a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:28.286754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd373-e294-49a4-8ed9-9c6375d3307a None admin] PATH_INFO: `/v3/services/network` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:28.287407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fd373-e294-49a4-8ed9-9c6375d3307a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:28.288794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fd373-e294-49a4-8ed9-9c6375d3307a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-fd1fd373-e294-49a4-8ed9-9c6375d3307a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:28.306206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:28.307196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:28.308017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 356/717] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:28 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:28.315875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435f3e8-ee49-4377-8849-30b02db1a312 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:28.450012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:28.577225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435f3e8-ee49-4377-8849-30b02db1a312 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:28.646853 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468a3a0-aa60-41b8-82c8-a6d58e8a4ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:28.647544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468a3a0-aa60-41b8-82c8-a6d58e8a4ced None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:28.647823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468a3a0-aa60-41b8-82c8-a6d58e8a4ced None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:28.648097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468a3a0-aa60-41b8-82c8-a6d58e8a4ced None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:28.648815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468a3a0-aa60-41b8-82c8-a6d58e8a4ced None admin] RBAC: Adding query filter params (name=network) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:28.649121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468a3a0-aa60-41b8-82c8-a6d58e8a4ced None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:28.650530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468a3a0-aa60-41b8-82c8-a6d58e8a4ced None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:28.659566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 362/718] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:40:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:28.665320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd373-e294-49a4-8ed9-9c6375d3307a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:28.771377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:28.828967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd373-e294-49a4-8ed9-9c6375d3307a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:28.877744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77924a-820d-49d4-b280-dcac7fa84093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:28.878419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77924a-820d-49d4-b280-dcac7fa84093 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:28.878701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77924a-820d-49d4-b280-dcac7fa84093 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:28.878981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77924a-820d-49d4-b280-dcac7fa84093 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:28.879696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77924a-820d-49d4-b280-dcac7fa84093 None admin] RBAC: Adding query filter params (type=network) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:28.880005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77924a-820d-49d4-b280-dcac7fa84093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:28.881382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77924a-820d-49d4-b280-dcac7fa84093 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:28.886325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 357/719] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:40:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:29.072552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d9cd-4185-4b23-96d3-95a9cd957d8e None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:29.073077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d9cd-4185-4b23-96d3-95a9cd957d8e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:29.073154 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d9cd-4185-4b23-96d3-95a9cd957d8e None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:29.074874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 363/720] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:29.082438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a26f6-d894-48cc-bc33-8fe19da2066d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:29.082859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a26f6-d894-48cc-bc33-8fe19da2066d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:29.083274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a26f6-d894-48cc-bc33-8fe19da2066d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:29.118190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-497a26f6-d894-48cc-bc33-8fe19da2066d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:29.119763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-497a26f6-d894-48cc-bc33-8fe19da2066d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:29.205735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 358/721] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:29.229510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4d9cd-4185-4b23-96d3-95a9cd957d8e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:29.350677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4d9cd-4185-4b23-96d3-95a9cd957d8e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:29.361848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:29.401520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fef3c3-9c67-4952-93db-0e31b63f47e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:29.402346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fef3c3-9c67-4952-93db-0e31b63f47e9 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:29.402627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fef3c3-9c67-4952-93db-0e31b63f47e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:29.402898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fef3c3-9c67-4952-93db-0e31b63f47e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:29.424634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fef3c3-9c67-4952-93db-0e31b63f47e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-f1fef3c3-9c67-4952-93db-0e31b63f47e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:29.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:29.433496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:29.434335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 364/722] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:29.440604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a26f6-d894-48cc-bc33-8fe19da2066d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:29.541993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:29.609565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a26f6-d894-48cc-bc33-8fe19da2066d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:29.653078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a633f5-e987-49fb-a553-767db1e7047c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:29.653773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a633f5-e987-49fb-a553-767db1e7047c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:29.654057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a633f5-e987-49fb-a553-767db1e7047c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:29.654333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a633f5-e987-49fb-a553-767db1e7047c None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:29.655088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a633f5-e987-49fb-a553-767db1e7047c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:29.655398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a633f5-e987-49fb-a553-767db1e7047c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:29.656835 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a633f5-e987-49fb-a553-767db1e7047c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:29.666339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 359/723] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:29.672163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fef3c3-9c67-4952-93db-0e31b63f47e9 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:29.782863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fef3c3-9c67-4952-93db-0e31b63f47e9 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:29.817767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b944fe4-b948-4f35-a73f-ab14b3a6d9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:29.818985 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944fe4-b948-4f35-a73f-ab14b3a6d9f0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:29.819370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944fe4-b948-4f35-a73f-ab14b3a6d9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:29.819655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944fe4-b948-4f35-a73f-ab14b3a6d9f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:29.831963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b944fe4-b948-4f35-a73f-ab14b3a6d9f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:29.834351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b944fe4-b948-4f35-a73f-ab14b3a6d9f0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-7b944fe4-b948-4f35-a73f-ab14b3a6d9f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:29.861689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:29.862689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:29.863568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:29.863568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:29.863568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:29.863568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:29.863568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 365/724] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:29.870133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a633f5-e987-49fb-a553-767db1e7047c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:29.974614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a633f5-e987-49fb-a553-767db1e7047c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:29.995972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4c7af-2cc4-4ece-8b43-3ae94ce7918b None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:29.996263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4c7af-2cc4-4ece-8b43-3ae94ce7918b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:29.996523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4c7af-2cc4-4ece-8b43-3ae94ce7918b None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:29.997960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:29.998697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 366/725] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:30.011030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb2a0b-4b9c-4027-bc5c-4d8c5308519d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.011333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb2a0b-4b9c-4027-bc5c-4d8c5308519d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.011592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb2a0b-4b9c-4027-bc5c-4d8c5308519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.025728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff709a4-6220-4b23-a1b9-8ee1090348dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:30.026400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff709a4-6220-4b23-a1b9-8ee1090348dd None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.026677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff709a4-6220-4b23-a1b9-8ee1090348dd None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.026944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff709a4-6220-4b23-a1b9-8ee1090348dd None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.027665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff709a4-6220-4b23-a1b9-8ee1090348dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:30.027962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff709a4-6220-4b23-a1b9-8ee1090348dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:30.029332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff709a4-6220-4b23-a1b9-8ee1090348dd None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:30.029527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-64bb2a0b-4b9c-4027-bc5c-4d8c5308519d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:30.030937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-64bb2a0b-4b9c-4027-bc5c-4d8c5308519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:30.049301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:30.053802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 360/726] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:30.064050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff709a4-6220-4b23-a1b9-8ee1090348dd None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:30.158444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 367/727] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:30.185370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52a689-1e53-473f-a341-4b5cbe0f7546 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.185370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52a689-1e53-473f-a341-4b5cbe0f7546 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.185370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52a689-1e53-473f-a341-4b5cbe0f7546 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.186178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 368/728] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:30.190705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc0c4a-f6cf-4382-8540-7965bfd69ee7 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.191208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc0c4a-f6cf-4382-8540-7965bfd69ee7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.191672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc0c4a-f6cf-4382-8540-7965bfd69ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.211297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-05cc0c4a-f6cf-4382-8540-7965bfd69ee7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:30.212830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-05cc0c4a-f6cf-4382-8540-7965bfd69ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:30.234574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff709a4-6220-4b23-a1b9-8ee1090348dd None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:30.270936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283baf4c-47e0-4c64-820d-e42cd1e15650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:30.271712 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283baf4c-47e0-4c64-820d-e42cd1e15650 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.272046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283baf4c-47e0-4c64-820d-e42cd1e15650 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.272326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283baf4c-47e0-4c64-820d-e42cd1e15650 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.279954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283baf4c-47e0-4c64-820d-e42cd1e15650 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:30.282422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283baf4c-47e0-4c64-820d-e42cd1e15650 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:30.291065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 369/729] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 19:40:30.292202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc0c4a-f6cf-4382-8540-7965bfd69ee7 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-283baf4c-47e0-4c64-820d-e42cd1e15650 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:30.307498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:30.308548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:30.309611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:30.309611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:30.309611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:30.309611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:30.309611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:30.309611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:30.309611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:30.309611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:30.309611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:30.309611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:30.313359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 361/730] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:30.318688 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283baf4c-47e0-4c64-820d-e42cd1e15650 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:30.371196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:30.388874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:30.412483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc0c4a-f6cf-4382-8540-7965bfd69ee7 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:30.440956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283baf4c-47e0-4c64-820d-e42cd1e15650 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:30.450754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159ee41-a043-42f0-9d51-2371a0b9de10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:30.451924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159ee41-a043-42f0-9d51-2371a0b9de10 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.452470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159ee41-a043-42f0-9d51-2371a0b9de10 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.453865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159ee41-a043-42f0-9d51-2371a0b9de10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.465659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159ee41-a043-42f0-9d51-2371a0b9de10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-c159ee41-a043-42f0-9d51-2371a0b9de10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:30.475165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:30.476130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:30.476969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:30.479661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 370/731] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:30.480738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159ee41-a043-42f0-9d51-2371a0b9de10 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:30.481952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616a3d8-923c-4646-b3b1-05203adc06ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:30.482854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616a3d8-923c-4646-b3b1-05203adc06ee None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.483348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616a3d8-923c-4646-b3b1-05203adc06ee None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.484421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616a3d8-923c-4646-b3b1-05203adc06ee None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.485503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616a3d8-923c-4646-b3b1-05203adc06ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:30.485903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616a3d8-923c-4646-b3b1-05203adc06ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:30.487416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616a3d8-923c-4646-b3b1-05203adc06ee None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:30.515865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 362/732] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 1 19:40:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:30.515865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616a3d8-923c-4646-b3b1-05203adc06ee None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:30.625215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159ee41-a043-42f0-9d51-2371a0b9de10 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:30.631214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:30.639084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616a3d8-923c-4646-b3b1-05203adc06ee None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:30.662574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6b2c3-83c6-4afd-84bf-b810f150514e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:30.663349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6b2c3-83c6-4afd-84bf-b810f150514e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.663636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6b2c3-83c6-4afd-84bf-b810f150514e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.663916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6b2c3-83c6-4afd-84bf-b810f150514e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.674852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6b2c3-83c6-4afd-84bf-b810f150514e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:30.678029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78c435-8ec1-416d-a19f-ad19765e02f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:30.678690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78c435-8ec1-416d-a19f-ad19765e02f6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.678971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78c435-8ec1-416d-a19f-ad19765e02f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.679246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78c435-8ec1-416d-a19f-ad19765e02f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.679983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78c435-8ec1-416d-a19f-ad19765e02f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:30.680291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78c435-8ec1-416d-a19f-ad19765e02f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:30.681689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78c435-8ec1-416d-a19f-ad19765e02f6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-0cc6b2c3-83c6-4afd-84bf-b810f150514e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:30.682708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:30.683592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:30.684481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 371/733] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:30.685185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6b2c3-83c6-4afd-84bf-b810f150514e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:30.691713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:30.701983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 363/734] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:30.701983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda9f2-9123-4d35-a33f-8b25163ed00f None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.701983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda9f2-9123-4d35-a33f-8b25163ed00f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.701983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda9f2-9123-4d35-a33f-8b25163ed00f None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.701983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 364/735] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:30.701983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda9f2-9123-4d35-a33f-8b25163ed00f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:30.800123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6b2c3-83c6-4afd-84bf-b810f150514e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:30.811635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda9f2-9123-4d35-a33f-8b25163ed00f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:30.836580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f800b07-fb5a-4588-b167-2dae5df8f3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:30.837347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f800b07-fb5a-4588-b167-2dae5df8f3a5 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.837652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f800b07-fb5a-4588-b167-2dae5df8f3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.837934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f800b07-fb5a-4588-b167-2dae5df8f3a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.848787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b9927-b7c0-488d-8233-bc84f79e4966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:30.849783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b9927-b7c0-488d-8233-bc84f79e4966 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:30.850171 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b9927-b7c0-488d-8233-bc84f79e4966 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:30.850455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b9927-b7c0-488d-8233-bc84f79e4966 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:30.851195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b9927-b7c0-488d-8233-bc84f79e4966 None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:30.851503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b9927-b7c0-488d-8233-bc84f79e4966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:30.852876 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b9927-b7c0-488d-8233-bc84f79e4966 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:30.867585 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 365/736] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:30.867585 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b9927-b7c0-488d-8233-bc84f79e4966 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:30.887642 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:30.913270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f800b07-fb5a-4588-b167-2dae5df8f3a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:30.914676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f800b07-fb5a-4588-b167-2dae5df8f3a5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:30.916574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:30.974416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b9927-b7c0-488d-8233-bc84f79e4966 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:31.010830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08737408-85ac-47fe-8e30-31705a8f18b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:31.011622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08737408-85ac-47fe-8e30-31705a8f18b8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.011905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08737408-85ac-47fe-8e30-31705a8f18b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.012180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08737408-85ac-47fe-8e30-31705a8f18b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.024295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 372/737] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 19:40:31.024295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0b3ef-ffc5-4b21-aaff-2cc744369663 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.024295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0b3ef-ffc5-4b21-aaff-2cc744369663 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.024295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0b3ef-ffc5-4b21-aaff-2cc744369663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.027589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08737408-85ac-47fe-8e30-31705a8f18b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:31.029150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08737408-85ac-47fe-8e30-31705a8f18b8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:31.038567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-02f0b3ef-ffc5-4b21-aaff-2cc744369663 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:31.039759 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-02f0b3ef-ffc5-4b21-aaff-2cc744369663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-08737408-85ac-47fe-8e30-31705a8f18b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:31.048156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:31.049037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:31.050319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:31.050319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:31.050319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:31.050319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:31.050319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 366/738] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:31.050779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08737408-85ac-47fe-8e30-31705a8f18b8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:31.121912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 373/739] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:31.121912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0b3ef-ffc5-4b21-aaff-2cc744369663 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:31.131143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:31.166313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08737408-85ac-47fe-8e30-31705a8f18b8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:31.170130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:31.205233 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72059c6-9679-4c28-8a29-3696439dd763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:31.206049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72059c6-9679-4c28-8a29-3696439dd763 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.206338 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72059c6-9679-4c28-8a29-3696439dd763 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.206612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72059c6-9679-4c28-8a29-3696439dd763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.218849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72059c6-9679-4c28-8a29-3696439dd763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:31.220489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72059c6-9679-4c28-8a29-3696439dd763 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-a72059c6-9679-4c28-8a29-3696439dd763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:31.238529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:31.239478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:31.240356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:31.240356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:31.240356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:31.240356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:31.240356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 367/740] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:31.240641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72059c6-9679-4c28-8a29-3696439dd763 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:31.252614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0b3ef-ffc5-4b21-aaff-2cc744369663 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:31.287384 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab00d7d-b1d2-40b6-9dda-46271b128a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:31.288198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab00d7d-b1d2-40b6-9dda-46271b128a09 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.288473 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab00d7d-b1d2-40b6-9dda-46271b128a09 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.288749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab00d7d-b1d2-40b6-9dda-46271b128a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.299972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab00d7d-b1d2-40b6-9dda-46271b128a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:31.301420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab00d7d-b1d2-40b6-9dda-46271b128a09 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-dab00d7d-b1d2-40b6-9dda-46271b128a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:31.319803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:31.320727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:31.321637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:31.321637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:31.321637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:31.321637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:31.321637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 374/741] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:31.321951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab00d7d-b1d2-40b6-9dda-46271b128a09 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:31.349073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72059c6-9679-4c28-8a29-3696439dd763 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:31.376597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:31.387878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619979c4-4a35-44af-b6f8-321f12d3ef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:31.388530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619979c4-4a35-44af-b6f8-321f12d3ef16 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.388807 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619979c4-4a35-44af-b6f8-321f12d3ef16 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.389085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619979c4-4a35-44af-b6f8-321f12d3ef16 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.389856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619979c4-4a35-44af-b6f8-321f12d3ef16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:31.390722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619979c4-4a35-44af-b6f8-321f12d3ef16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:31.392249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619979c4-4a35-44af-b6f8-321f12d3ef16 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:31.403864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 368/742] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:31.404435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619979c4-4a35-44af-b6f8-321f12d3ef16 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:31.408943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:31.425334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab00d7d-b1d2-40b6-9dda-46271b128a09 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:31.460487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d4b16-d5d3-431e-bbfc-00fdc39e50d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:31.461140 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d4b16-d5d3-431e-bbfc-00fdc39e50d8 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.461419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d4b16-d5d3-431e-bbfc-00fdc39e50d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.461735 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d4b16-d5d3-431e-bbfc-00fdc39e50d8 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.462438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d4b16-d5d3-431e-bbfc-00fdc39e50d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:31.463816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d4b16-d5d3-431e-bbfc-00fdc39e50d8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:31.478935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 375/743] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:40:31 2022] POST /identity/v3/services => generated 251 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:31.478935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d4b16-d5d3-431e-bbfc-00fdc39e50d8 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:31.522877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619979c4-4a35-44af-b6f8-321f12d3ef16 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:31.580451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e1b9f-afca-46c5-841f-3dd03d004f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:31.581249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e1b9f-afca-46c5-841f-3dd03d004f71 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.581532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e1b9f-afca-46c5-841f-3dd03d004f71 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.581909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e1b9f-afca-46c5-841f-3dd03d004f71 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.582695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e1b9f-afca-46c5-841f-3dd03d004f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:31.583046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e1b9f-afca-46c5-841f-3dd03d004f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:31.584556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e1b9f-afca-46c5-841f-3dd03d004f71 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:31.598244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 369/744] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:31.604172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e994f21-ec2a-43cc-aefa-1bf9b046fc8b None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.604464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e994f21-ec2a-43cc-aefa-1bf9b046fc8b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.604739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e994f21-ec2a-43cc-aefa-1bf9b046fc8b None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.606323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 370/745] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:31.606886 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e994f21-ec2a-43cc-aefa-1bf9b046fc8b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:31.611095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d4b16-d5d3-431e-bbfc-00fdc39e50d8 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:31.651661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedc3fd-b2c0-4e93-b764-d711403e52ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:31.652327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedc3fd-b2c0-4e93-b764-d711403e52ee None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.652610 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedc3fd-b2c0-4e93-b764-d711403e52ee None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.652882 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedc3fd-b2c0-4e93-b764-d711403e52ee None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.653741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedc3fd-b2c0-4e93-b764-d711403e52ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:31.654050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedc3fd-b2c0-4e93-b764-d711403e52ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:31.655423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedc3fd-b2c0-4e93-b764-d711403e52ee None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:31.659812 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:31.671976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 376/746] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:31.671976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedc3fd-b2c0-4e93-b764-d711403e52ee None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:31.676780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:31.731819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e994f21-ec2a-43cc-aefa-1bf9b046fc8b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:31.768087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd6ff2-a488-4e34-ab51-cddbc0244ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:31.768878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd6ff2-a488-4e34-ab51-cddbc0244ea8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.769158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd6ff2-a488-4e34-ab51-cddbc0244ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.769431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd6ff2-a488-4e34-ab51-cddbc0244ea8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.777254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd6ff2-a488-4e34-ab51-cddbc0244ea8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:31.778759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd6ff2-a488-4e34-ab51-cddbc0244ea8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:31.783395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedc3fd-b2c0-4e93-b764-d711403e52ee None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-4fbd6ff2-a488-4e34-ab51-cddbc0244ea8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:31.794180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:31.795298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:31.796556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:31.796556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:31.796556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:31.796556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:31.796556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:31.796556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:31.796556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:31.796556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:31.796556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:31.796556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:31.800047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 371/747] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 1 19:40:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:31.801351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fee9c-7d04-4fef-8514-80ab56d3ace5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.801656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fee9c-7d04-4fef-8514-80ab56d3ace5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.801936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fee9c-7d04-4fef-8514-80ab56d3ace5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.818326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-212fee9c-7d04-4fef-8514-80ab56d3ace5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:31.819534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-212fee9c-7d04-4fef-8514-80ab56d3ace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:31.824878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d2953-f34f-412f-b7e1-821ea859c711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:31.825981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d2953-f34f-412f-b7e1-821ea859c711 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:31.826477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d2953-f34f-412f-b7e1-821ea859c711 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:31.826968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d2953-f34f-412f-b7e1-821ea859c711 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:31.835803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d2953-f34f-412f-b7e1-821ea859c711 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:31.837287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d2953-f34f-412f-b7e1-821ea859c711 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6c0d2953-f34f-412f-b7e1-821ea859c711 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:31.852751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:31.853795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 377/748] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 1 19:40:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:31.854734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d2953-f34f-412f-b7e1-821ea859c711 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:31.898119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 372/749] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 19:40:31.899085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fee9c-7d04-4fef-8514-80ab56d3ace5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:31.935713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:31.940747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:31.986076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d2953-f34f-412f-b7e1-821ea859c711 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:32.018418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fee9c-7d04-4fef-8514-80ab56d3ace5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:32.024696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fcc42-b8f5-486c-8ce5-22fcf1a8019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:32.025624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fcc42-b8f5-486c-8ce5-22fcf1a8019e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:32.026422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fcc42-b8f5-486c-8ce5-22fcf1a8019e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:32.026800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fcc42-b8f5-486c-8ce5-22fcf1a8019e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:32.034936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fcc42-b8f5-486c-8ce5-22fcf1a8019e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:32.036668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fcc42-b8f5-486c-8ce5-22fcf1a8019e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-371fcc42-b8f5-486c-8ce5-22fcf1a8019e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:32.051822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:32.052713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:32.054133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:32.054133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:32.054133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:32.054133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:32.054133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:32.054133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:32.054133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:32.054133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:32.055251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f9ec4-2ce2-461d-8276-472178a9dcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:32.055920 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f9ec4-2ce2-461d-8276-472178a9dcd4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:32.056194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f9ec4-2ce2-461d-8276-472178a9dcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:32.056464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f9ec4-2ce2-461d-8276-472178a9dcd4 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:32.057292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f9ec4-2ce2-461d-8276-472178a9dcd4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:32.057793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f9ec4-2ce2-461d-8276-472178a9dcd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:32.057891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 378/750] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:32.058729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fcc42-b8f5-486c-8ce5-22fcf1a8019e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:32.059113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f9ec4-2ce2-461d-8276-472178a9dcd4 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:32.082589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 373/751] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 1 19:40:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:32.082589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f9ec4-2ce2-461d-8276-472178a9dcd4 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:32.166188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fcc42-b8f5-486c-8ce5-22fcf1a8019e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:32.177781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:32.192739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f9ec4-2ce2-461d-8276-472178a9dcd4 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:32.194383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:32.206429 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a20d-4e82-44cc-94c3-3a1254d13e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:32.207104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a20d-4e82-44cc-94c3-3a1254d13e94 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:32.207382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a20d-4e82-44cc-94c3-3a1254d13e94 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:32.207653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a20d-4e82-44cc-94c3-3a1254d13e94 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:32.208498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a20d-4e82-44cc-94c3-3a1254d13e94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:32.208821 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a20d-4e82-44cc-94c3-3a1254d13e94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:32.210284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a20d-4e82-44cc-94c3-3a1254d13e94 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:32.233988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 379/752] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 1 19:40:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:32.233988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a20d-4e82-44cc-94c3-3a1254d13e94 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:32.238627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c8d0c-71fe-48eb-a227-f5b9500e01d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:32.239384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c8d0c-71fe-48eb-a227-f5b9500e01d7 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:32.239665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c8d0c-71fe-48eb-a227-f5b9500e01d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:32.239938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c8d0c-71fe-48eb-a227-f5b9500e01d7 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:32.240670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c8d0c-71fe-48eb-a227-f5b9500e01d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:32.242083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c8d0c-71fe-48eb-a227-f5b9500e01d7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:32.265789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 374/753] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:40:32 2022] POST /identity/v3/services => generated 240 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:32.267110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c8d0c-71fe-48eb-a227-f5b9500e01d7 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:32.411428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c8d0c-71fe-48eb-a227-f5b9500e01d7 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:32.457339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90760820-b5b1-4b11-8bc0-8c224b5345b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:32.458086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90760820-b5b1-4b11-8bc0-8c224b5345b7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:32.458359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90760820-b5b1-4b11-8bc0-8c224b5345b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:32.459036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90760820-b5b1-4b11-8bc0-8c224b5345b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:32.466048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a20d-4e82-44cc-94c3-3a1254d13e94 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:32.470321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90760820-b5b1-4b11-8bc0-8c224b5345b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:32.471810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90760820-b5b1-4b11-8bc0-8c224b5345b7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:32.476760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-90760820-b5b1-4b11-8bc0-8c224b5345b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:32.496361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:32.497282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:32.498882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:32.498882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:32.498882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:32.498882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:32.498882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 375/754] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:32.498882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90760820-b5b1-4b11-8bc0-8c224b5345b7 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:32.516902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54b323-ddc8-4dbf-87e8-764d3a03042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:32.517984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54b323-ddc8-4dbf-87e8-764d3a03042c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:32.518452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54b323-ddc8-4dbf-87e8-764d3a03042c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:32.518936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54b323-ddc8-4dbf-87e8-764d3a03042c None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:32.520045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54b323-ddc8-4dbf-87e8-764d3a03042c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:32.520571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54b323-ddc8-4dbf-87e8-764d3a03042c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:32.522264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54b323-ddc8-4dbf-87e8-764d3a03042c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:32.540177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 380/755] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:32.541370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54b323-ddc8-4dbf-87e8-764d3a03042c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:32.601535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:32.618266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90760820-b5b1-4b11-8bc0-8c224b5345b7 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:32.656772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b65b17-2fd3-44df-a931-78f3bf281105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:32.657415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b65b17-2fd3-44df-a931-78f3bf281105 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:32.657738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b65b17-2fd3-44df-a931-78f3bf281105 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:32.658009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b65b17-2fd3-44df-a931-78f3bf281105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:32.689730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54b323-ddc8-4dbf-87e8-764d3a03042c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:32.726266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1fbfd-1490-4519-bb22-da3a5fc7f85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:32.727221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:32.728530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1fbfd-1490-4519-bb22-da3a5fc7f85a None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:32.729036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1fbfd-1490-4519-bb22-da3a5fc7f85a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:32.729524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1fbfd-1490-4519-bb22-da3a5fc7f85a None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:32.730546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1fbfd-1490-4519-bb22-da3a5fc7f85a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:32.731047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1fbfd-1490-4519-bb22-da3a5fc7f85a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:32.732994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1fbfd-1490-4519-bb22-da3a5fc7f85a None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:32.734975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b65b17-2fd3-44df-a931-78f3bf281105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:32.737020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b65b17-2fd3-44df-a931-78f3bf281105 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:32.748056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 381/756] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:32.749208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1fbfd-1490-4519-bb22-da3a5fc7f85a None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:32.856324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 376/757] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:40:32.857607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b65b17-2fd3-44df-a931-78f3bf281105 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:32.870703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1fbfd-1490-4519-bb22-da3a5fc7f85a None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:32.873637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:32.917542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12215023-25c8-435b-bd56-6e63f6135895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:32.918561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215023-25c8-435b-bd56-6e63f6135895 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:32.919023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215023-25c8-435b-bd56-6e63f6135895 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:32.919456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12215023-25c8-435b-bd56-6e63f6135895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:32.920430 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12215023-25c8-435b-bd56-6e63f6135895 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=c2bce9daec8c446d89eb6cc5950d6649) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:32.920890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12215023-25c8-435b-bd56-6e63f6135895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:32.923202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12215023-25c8-435b-bd56-6e63f6135895 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:32.937913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 382/758] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:32 2022] GET /identity/v3/role_assignments?user.id=c2bce9daec8c446d89eb6cc5950d6649&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:32.939250 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12215023-25c8-435b-bd56-6e63f6135895 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:32.971021 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:32.979923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b65b17-2fd3-44df-a931-78f3bf281105 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:33.060502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a9836-7d9c-4719-95e6-3c1d6ddfd0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:33.062607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a9836-7d9c-4719-95e6-3c1d6ddfd0d4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:33.063042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a9836-7d9c-4719-95e6-3c1d6ddfd0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:33.063465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a9836-7d9c-4719-95e6-3c1d6ddfd0d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:33.081111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a9836-7d9c-4719-95e6-3c1d6ddfd0d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:33.082862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a9836-7d9c-4719-95e6-3c1d6ddfd0d4 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-702a9836-7d9c-4719-95e6-3c1d6ddfd0d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:33.099631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:33.100771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:33.101893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:33.101893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:33.101893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:33.101893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:33.101893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:33.101893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:33.101893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:33.101893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:33.102395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12215023-25c8-435b-bd56-6e63f6135895 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:33.104212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 377/759] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:33.114827 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a9836-7d9c-4719-95e6-3c1d6ddfd0d4 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:33.152455 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61554a12-07cf-40a3-b76e-237aa09811fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:33.153315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61554a12-07cf-40a3-b76e-237aa09811fe None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:33.153607 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61554a12-07cf-40a3-b76e-237aa09811fe None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:33.153885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61554a12-07cf-40a3-b76e-237aa09811fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:33.172347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61554a12-07cf-40a3-b76e-237aa09811fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:33.173817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61554a12-07cf-40a3-b76e-237aa09811fe None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-61554a12-07cf-40a3-b76e-237aa09811fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:33.200871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:33.202325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:33.203389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:33.203389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:33.203389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:33.203389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:33.203389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 383/760] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:33.209480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61554a12-07cf-40a3-b76e-237aa09811fe None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:33.241664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:33.261962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a9836-7d9c-4719-95e6-3c1d6ddfd0d4 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:33.304347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef7724-187b-4c20-bbd3-7de9b5c7fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:33.305053 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef7724-187b-4c20-bbd3-7de9b5c7fb6e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:33.305326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef7724-187b-4c20-bbd3-7de9b5c7fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:33.307318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef7724-187b-4c20-bbd3-7de9b5c7fb6e None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:33.308117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef7724-187b-4c20-bbd3-7de9b5c7fb6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:33.308423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef7724-187b-4c20-bbd3-7de9b5c7fb6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:33.309893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef7724-187b-4c20-bbd3-7de9b5c7fb6e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:33.341322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 378/761] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:33.347696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef7724-187b-4c20-bbd3-7de9b5c7fb6e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:33.388216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61554a12-07cf-40a3-b76e-237aa09811fe None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:33.395306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:33.431299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78983fbd-0d14-47ff-a0bb-0d973af0a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:33.433219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983fbd-0d14-47ff-a0bb-0d973af0a036 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:33.433503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983fbd-0d14-47ff-a0bb-0d973af0a036 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:33.434691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78983fbd-0d14-47ff-a0bb-0d973af0a036 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:33.435423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78983fbd-0d14-47ff-a0bb-0d973af0a036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:33.435724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78983fbd-0d14-47ff-a0bb-0d973af0a036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:33.441365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78983fbd-0d14-47ff-a0bb-0d973af0a036 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:33.468108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 384/762] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:33.473956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78983fbd-0d14-47ff-a0bb-0d973af0a036 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:33.529437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef7724-187b-4c20-bbd3-7de9b5c7fb6e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:33.575506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d005853-a675-4f88-b506-1c4338278078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:33.578728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d005853-a675-4f88-b506-1c4338278078 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:33.579016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d005853-a675-4f88-b506-1c4338278078 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:33.579290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d005853-a675-4f88-b506-1c4338278078 None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/ad06f31859f9413f9db6039ac2abf982/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:33.586434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d005853-a675-4f88-b506-1c4338278078 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=ad06f31859f9413f9db6039ac2abf982, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:33.587898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d005853-a675-4f88-b506-1c4338278078 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:33.617024 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 379/763] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:40:33 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/ad06f31859f9413f9db6039ac2abf982/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:40:33.644041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:33.667945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78983fbd-0d14-47ff-a0bb-0d973af0a036 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:33.707973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818fb20-7549-4006-a7c5-ac9233374e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:33.708851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818fb20-7549-4006-a7c5-ac9233374e49 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:33.709185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818fb20-7549-4006-a7c5-ac9233374e49 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:33.709464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818fb20-7549-4006-a7c5-ac9233374e49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:33.717793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818fb20-7549-4006-a7c5-ac9233374e49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:33.719979 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818fb20-7549-4006-a7c5-ac9233374e49 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:33.726572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5dd89-9f94-4048-9a80-8f9b66a60f35 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:33.727191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5dd89-9f94-4048-9a80-8f9b66a60f35 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:33.727727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5dd89-9f94-4048-9a80-8f9b66a60f35 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:33.733095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 380/764] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:33.735220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2424b3e-1083-404b-add3-f8b31b56f089 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:33.735770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2424b3e-1083-404b-add3-f8b31b56f089 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-5818fb20-7549-4006-a7c5-ac9233374e49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:33.735963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:33.736794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:33.739122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:33.739122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:33.739122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:33.739122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:33.739122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:33.739122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:33.739122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:33.739122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:33.739122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 385/765] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:33.740222 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2424b3e-1083-404b-add3-f8b31b56f089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:33.747586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818fb20-7549-4006-a7c5-ac9233374e49 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:33.778767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c2424b3e-1083-404b-add3-f8b31b56f089 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:33.780466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c2424b3e-1083-404b-add3-f8b31b56f089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:33.859857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818fb20-7549-4006-a7c5-ac9233374e49 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:33.879268 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 381/766] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:33.896022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705016c-a0af-41df-9eeb-82890ca87c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:33.896680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705016c-a0af-41df-9eeb-82890ca87c8d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:33.896960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705016c-a0af-41df-9eeb-82890ca87c8d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:33.897235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705016c-a0af-41df-9eeb-82890ca87c8d None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:33.898109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705016c-a0af-41df-9eeb-82890ca87c8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:33.898418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705016c-a0af-41df-9eeb-82890ca87c8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:33.899898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705016c-a0af-41df-9eeb-82890ca87c8d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:33.906859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2424b3e-1083-404b-add3-f8b31b56f089 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:33.911023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:33.914906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 386/767] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:33.919778 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:33.921959 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705016c-a0af-41df-9eeb-82890ca87c8d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:34.035029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705016c-a0af-41df-9eeb-82890ca87c8d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:34.076328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b2d63-b49d-4aef-8857-59c0ba503bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:34.077405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b2d63-b49d-4aef-8857-59c0ba503bff None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:34.078085 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b2d63-b49d-4aef-8857-59c0ba503bff None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:34.078543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b2d63-b49d-4aef-8857-59c0ba503bff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:34.079605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b2d63-b49d-4aef-8857-59c0ba503bff None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=cfa25a63dd6f4ad08b0f858356528284) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:34.080104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b2d63-b49d-4aef-8857-59c0ba503bff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:34.081833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b2d63-b49d-4aef-8857-59c0ba503bff None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:34.088937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 387/768] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:33 2022] GET /identity/v3/role_assignments?user.id=cfa25a63dd6f4ad08b0f858356528284&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:34.148301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2424b3e-1083-404b-add3-f8b31b56f089 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:34.188066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f92f5-5481-4352-a110-6d2d8134d369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:34.188905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f92f5-5481-4352-a110-6d2d8134d369 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:34.189177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f92f5-5481-4352-a110-6d2d8134d369 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:34.189442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f92f5-5481-4352-a110-6d2d8134d369 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:34.190153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f92f5-5481-4352-a110-6d2d8134d369 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:34.191576 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f92f5-5481-4352-a110-6d2d8134d369 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-d71f92f5-5481-4352-a110-6d2d8134d369 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:34.205511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:34.206548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:34.207366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 382/769] 10.222.0.66 () {56 vars in 1129 bytes} [Fri Jul 1 19:40:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:34.213350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b2d63-b49d-4aef-8857-59c0ba503bff None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:34.299145 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:34.353404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b2d63-b49d-4aef-8857-59c0ba503bff None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:34.398345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b1280-220a-4993-b5c3-2da383b77d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:34.399037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b1280-220a-4993-b5c3-2da383b77d68 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:34.399319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b1280-220a-4993-b5c3-2da383b77d68 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:34.399594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b1280-220a-4993-b5c3-2da383b77d68 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:34.400321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b1280-220a-4993-b5c3-2da383b77d68 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:34.400628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b1280-220a-4993-b5c3-2da383b77d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:34.402066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b1280-220a-4993-b5c3-2da383b77d68 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:34.414568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 388/770] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 1 19:40:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:34.419794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f92f5-5481-4352-a110-6d2d8134d369 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:34.513939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:34.567180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f92f5-5481-4352-a110-6d2d8134d369 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:34.596128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74bc6a-835c-48b7-ba5a-b8871abc7bc8 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:34.596411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74bc6a-835c-48b7-ba5a-b8871abc7bc8 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:34.596687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74bc6a-835c-48b7-ba5a-b8871abc7bc8 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:34.598332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 389/771] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:34.604260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1cd0f-77ce-4edf-89b3-6fa4bc22ed6d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:34.604547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1cd0f-77ce-4edf-89b3-6fa4bc22ed6d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:34.604825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1cd0f-77ce-4edf-89b3-6fa4bc22ed6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:34.626340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7e5f5-1f42-448b-b41d-0eeb4d573b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:34.627409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7e5f5-1f42-448b-b41d-0eeb4d573b33 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:34.627887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7e5f5-1f42-448b-b41d-0eeb4d573b33 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:34.628426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7e5f5-1f42-448b-b41d-0eeb4d573b33 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:34.629320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7e5f5-1f42-448b-b41d-0eeb4d573b33 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:34.629859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7e5f5-1f42-448b-b41d-0eeb4d573b33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:34.631418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7e5f5-1f42-448b-b41d-0eeb4d573b33 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:34.635047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-84a1cd0f-77ce-4edf-89b3-6fa4bc22ed6d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:34.636659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-84a1cd0f-77ce-4edf-89b3-6fa4bc22ed6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:34.643649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 383/772] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 1 19:40:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:34.650288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7e5f5-1f42-448b-b41d-0eeb4d573b33 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:34.763606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 390/773] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:34.799387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7e5f5-1f42-448b-b41d-0eeb4d573b33 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:34.807028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1cd0f-77ce-4edf-89b3-6fa4bc22ed6d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:34.872012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c86eba-0488-4c66-bdd4-f13432b27d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:34.872743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c86eba-0488-4c66-bdd4-f13432b27d87 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:34.873028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c86eba-0488-4c66-bdd4-f13432b27d87 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:34.873300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c86eba-0488-4c66-bdd4-f13432b27d87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:34.874663 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c86eba-0488-4c66-bdd4-f13432b27d87 None admin] RBAC: Adding query filter params (interface=public, service_id=607d89b56b874129ae65b5c44c96a617, region_id=RegionOne) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:34.874981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c86eba-0488-4c66-bdd4-f13432b27d87 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:34.876472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c86eba-0488-4c66-bdd4-f13432b27d87 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:34.882566 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:34.893571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 384/774] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 1 19:40:34 2022] GET /identity/v3/endpoints?service_id=607d89b56b874129ae65b5c44c96a617&interface=public®ion_id=RegionOne => generated 194 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:34.900118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c86eba-0488-4c66-bdd4-f13432b27d87 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:34.906445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:34.987572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1cd0f-77ce-4edf-89b3-6fa4bc22ed6d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:35.038265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6dc17-7716-4038-94cd-1fb807b279a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:35.039069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6dc17-7716-4038-94cd-1fb807b279a7 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:35.039350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6dc17-7716-4038-94cd-1fb807b279a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:35.039624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6dc17-7716-4038-94cd-1fb807b279a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:35.040282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6dc17-7716-4038-94cd-1fb807b279a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:35.041680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6dc17-7716-4038-94cd-1fb807b279a7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-f5c6dc17-7716-4038-94cd-1fb807b279a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:35.066164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:35.067188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 19:40:35.067998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:35.072352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 391/775] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:34 2022] GET /identity/v3/services/network => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:35.073535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6dc17-7716-4038-94cd-1fb807b279a7 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:35.128480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c86eba-0488-4c66-bdd4-f13432b27d87 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:35.176090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7cb4b-2e0f-4d59-b9fb-92ce9eb4842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:35.176761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7cb4b-2e0f-4d59-b9fb-92ce9eb4842d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:35.177017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7cb4b-2e0f-4d59-b9fb-92ce9eb4842d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:35.177282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7cb4b-2e0f-4d59-b9fb-92ce9eb4842d None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:35.177991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7cb4b-2e0f-4d59-b9fb-92ce9eb4842d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:35.179358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7cb4b-2e0f-4d59-b9fb-92ce9eb4842d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:35.193960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 385/776] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 1 19:40:34 2022] GET /identity/v3/services => generated 772 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:35.254260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6dc17-7716-4038-94cd-1fb807b279a7 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:35.282878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:35.307071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658e14a-7068-4105-98b4-62c969488171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:35.307762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e14a-7068-4105-98b4-62c969488171 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:35.308046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e14a-7068-4105-98b4-62c969488171 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:35.308319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e14a-7068-4105-98b4-62c969488171 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:35.309043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658e14a-7068-4105-98b4-62c969488171 None admin] RBAC: Adding query filter params (name=network) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:35.309342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658e14a-7068-4105-98b4-62c969488171 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:35.310776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658e14a-7068-4105-98b4-62c969488171 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:35.318434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 392/777] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:40:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:35.324249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7cb4b-2e0f-4d59-b9fb-92ce9eb4842d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:35.401552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:35.448056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7cb4b-2e0f-4d59-b9fb-92ce9eb4842d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:35.491686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87695dd9-7963-4c13-bd58-060659c29818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:35.492362 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87695dd9-7963-4c13-bd58-060659c29818 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:35.492638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87695dd9-7963-4c13-bd58-060659c29818 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:35.492905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87695dd9-7963-4c13-bd58-060659c29818 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:35.493697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87695dd9-7963-4c13-bd58-060659c29818 None admin] RBAC: Adding query filter params (type=network) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:35.494003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87695dd9-7963-4c13-bd58-060659c29818 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:35.495380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87695dd9-7963-4c13-bd58-060659c29818 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:35.501403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 386/778] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:40:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:35.510723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658e14a-7068-4105-98b4-62c969488171 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:35.608513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a2ffa-eb17-40f4-9791-57386cd69e54 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:35.608963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a2ffa-eb17-40f4-9791-57386cd69e54 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:35.609089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a2ffa-eb17-40f4-9791-57386cd69e54 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:35.610707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 387/779] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:35.615863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a133b-ed15-43ab-a3f1-d7f6bd455ba2 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:35.616141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a133b-ed15-43ab-a3f1-d7f6bd455ba2 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:35.616457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a133b-ed15-43ab-a3f1-d7f6bd455ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:35.635017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e82a133b-ed15-43ab-a3f1-d7f6bd455ba2 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:35.636252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e82a133b-ed15-43ab-a3f1-d7f6bd455ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:35.721109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658e14a-7068-4105-98b4-62c969488171 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:35.754710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 388/780] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:35.782320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6c298-3fe6-4007-8bb3-7757a6802a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:35.783046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6c298-3fe6-4007-8bb3-7757a6802a99 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:35.783328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6c298-3fe6-4007-8bb3-7757a6802a99 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:35.783599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6c298-3fe6-4007-8bb3-7757a6802a99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:35.784441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6c298-3fe6-4007-8bb3-7757a6802a99 None admin] RBAC: Adding query filter params (interface=public, service_id=b08f22f4fe074aee9adf93e35ea421c3, region_id=RegionOne) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:35.784753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6c298-3fe6-4007-8bb3-7757a6802a99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:35.786433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6c298-3fe6-4007-8bb3-7757a6802a99 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:35.794518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 393/781] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 1 19:40:35 2022] GET /identity/v3/endpoints?service_id=b08f22f4fe074aee9adf93e35ea421c3&interface=public®ion_id=RegionOne => generated 194 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:35.797101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a133b-ed15-43ab-a3f1-d7f6bd455ba2 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:35.808519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6c298-3fe6-4007-8bb3-7757a6802a99 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:35.881650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:35.901751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:35.932776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a133b-ed15-43ab-a3f1-d7f6bd455ba2 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:35.970735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10a59a-5a8a-4ca4-acb6-aae4897a1eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:35.972017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10a59a-5a8a-4ca4-acb6-aae4897a1eb5 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:35.972498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10a59a-5a8a-4ca4-acb6-aae4897a1eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:35.973043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10a59a-5a8a-4ca4-acb6-aae4897a1eb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:35.996088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10a59a-5a8a-4ca4-acb6-aae4897a1eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-7a10a59a-5a8a-4ca4-acb6-aae4897a1eb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:36.005312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:36.006262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:36.007071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:36.007694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6c298-3fe6-4007-8bb3-7757a6802a99 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:36.010070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 389/782] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:36.016036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10a59a-5a8a-4ca4-acb6-aae4897a1eb5 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:36.076672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a355a-77eb-4c06-8dcf-46cdb3126029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:36.077381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a355a-77eb-4c06-8dcf-46cdb3126029 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.077679 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a355a-77eb-4c06-8dcf-46cdb3126029 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.077951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a355a-77eb-4c06-8dcf-46cdb3126029 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.078635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a355a-77eb-4c06-8dcf-46cdb3126029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:36.080004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a355a-77eb-4c06-8dcf-46cdb3126029 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:36.089343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 394/783] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 1 19:40:35 2022] GET /identity/v3/services => generated 772 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:36.152944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10a59a-5a8a-4ca4-acb6-aae4897a1eb5 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:36.171668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:36.198825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffedc37-26cd-4f98-90a0-8ba299c7f8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:36.200302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffedc37-26cd-4f98-90a0-8ba299c7f8d9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.200785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffedc37-26cd-4f98-90a0-8ba299c7f8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.201257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffedc37-26cd-4f98-90a0-8ba299c7f8d9 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.202899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffedc37-26cd-4f98-90a0-8ba299c7f8d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:36.203593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffedc37-26cd-4f98-90a0-8ba299c7f8d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:36.205220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffedc37-26cd-4f98-90a0-8ba299c7f8d9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:36.226883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 390/784] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:36.232841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a355a-77eb-4c06-8dcf-46cdb3126029 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:36.298531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ca92a-2538-4038-849c-72c80b894c30 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.299250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ca92a-2538-4038-849c-72c80b894c30 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.299740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ca92a-2538-4038-849c-72c80b894c30 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.301726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 391/785] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:36.307154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaf466-92ce-4ffe-ba6c-90e0829a243c None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.307703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaf466-92ce-4ffe-ba6c-90e0829a243c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.308195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaf466-92ce-4ffe-ba6c-90e0829a243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.327275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-1eeaf466-92ce-4ffe-ba6c-90e0829a243c None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:36.328919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1eeaf466-92ce-4ffe-ba6c-90e0829a243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:36.389333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a355a-77eb-4c06-8dcf-46cdb3126029 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:36.412719 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 392/786] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:36.419870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:36.441269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaf466-92ce-4ffe-ba6c-90e0829a243c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:36.448442 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb42315-ab39-4a4a-b181-400e8bea77fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:36.449359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb42315-ab39-4a4a-b181-400e8bea77fe None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.449761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb42315-ab39-4a4a-b181-400e8bea77fe None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.450109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb42315-ab39-4a4a-b181-400e8bea77fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.461682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb42315-ab39-4a4a-b181-400e8bea77fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:36.463223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb42315-ab39-4a4a-b181-400e8bea77fe None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-beb42315-ab39-4a4a-b181-400e8bea77fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:36.483657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:36.484626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:36.485497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:36.485497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:36.485497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:36.485497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:36.485497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 395/787] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:36.490824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb42315-ab39-4a4a-b181-400e8bea77fe None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:36.564994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaf466-92ce-4ffe-ba6c-90e0829a243c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:36.577195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:36.620524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f474e37-1cb1-4f09-b6f9-3b91c8f20730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:36.621511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f474e37-1cb1-4f09-b6f9-3b91c8f20730 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.621908 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f474e37-1cb1-4f09-b6f9-3b91c8f20730 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.622278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f474e37-1cb1-4f09-b6f9-3b91c8f20730 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.642260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb42315-ab39-4a4a-b181-400e8bea77fe None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:36.654508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f474e37-1cb1-4f09-b6f9-3b91c8f20730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-9f474e37-1cb1-4f09-b6f9-3b91c8f20730 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:36.671499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:36.672662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:36.673514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:36.674178 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 393/788] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:36.679701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f474e37-1cb1-4f09-b6f9-3b91c8f20730 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:36.682230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f7516-a63d-481d-8be9-e9120ebb4160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:36.684826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f7516-a63d-481d-8be9-e9120ebb4160 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.685286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f7516-a63d-481d-8be9-e9120ebb4160 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.685758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f7516-a63d-481d-8be9-e9120ebb4160 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.688171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f7516-a63d-481d-8be9-e9120ebb4160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:36.688680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f7516-a63d-481d-8be9-e9120ebb4160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:36.695981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f7516-a63d-481d-8be9-e9120ebb4160 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:36.718320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 396/789] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:36.724804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f7516-a63d-481d-8be9-e9120ebb4160 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:36.778015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:36.849114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f474e37-1cb1-4f09-b6f9-3b91c8f20730 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:36.870192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f7516-a63d-481d-8be9-e9120ebb4160 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:36.890670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667884a-b33e-4739-b5f3-e1bfc318c436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:36.891622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667884a-b33e-4739-b5f3-e1bfc318c436 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.892492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667884a-b33e-4739-b5f3-e1bfc318c436 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.892943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667884a-b33e-4739-b5f3-e1bfc318c436 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.894219 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667884a-b33e-4739-b5f3-e1bfc318c436 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:36.894851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667884a-b33e-4739-b5f3-e1bfc318c436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:36.896973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667884a-b33e-4739-b5f3-e1bfc318c436 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:36.906380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb7a01-6a92-40d7-8580-cf2b8d3c7fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:36.907146 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb7a01-6a92-40d7-8580-cf2b8d3c7fe1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.907432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb7a01-6a92-40d7-8580-cf2b8d3c7fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.907718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb7a01-6a92-40d7-8580-cf2b8d3c7fe1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.909708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 394/790] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:36.911964 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7244a4-0570-46b0-910f-a28e7e5a4bba None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.912431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7244a4-0570-46b0-910f-a28e7e5a4bba None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.912897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7244a4-0570-46b0-910f-a28e7e5a4bba None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.914617 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 395/791] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:36.915831 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7244a4-0570-46b0-910f-a28e7e5a4bba None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:36.916204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb7a01-6a92-40d7-8580-cf2b8d3c7fe1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:36.917690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb7a01-6a92-40d7-8580-cf2b8d3c7fe1 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-cdeb7a01-6a92-40d7-8580-cf2b8d3c7fe1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:36.941976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:36.942923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:36.943841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:36.943841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:36.943841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:36.943841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:36.943841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:36.943841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:36.943841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:36.943841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:36.943841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:36.943841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:36.946431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 397/792] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:36.947799 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f648939-f37b-4b43-8c5e-3ea1a12a155a None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:36.948234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f648939-f37b-4b43-8c5e-3ea1a12a155a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:36.948635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f648939-f37b-4b43-8c5e-3ea1a12a155a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:36.961979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:36.971174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-4f648939-f37b-4b43-8c5e-3ea1a12a155a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:36.972949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4f648939-f37b-4b43-8c5e-3ea1a12a155a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:37.038665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7244a4-0570-46b0-910f-a28e7e5a4bba None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:37.047826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 398/793] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:37.048957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f648939-f37b-4b43-8c5e-3ea1a12a155a None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:37.065940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:37.081773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ccfc8-bf9e-4d68-a173-3a6f79452de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:37.081773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ccfc8-bf9e-4d68-a173-3a6f79452de3 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.081773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ccfc8-bf9e-4d68-a173-3a6f79452de3 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.081773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ccfc8-bf9e-4d68-a173-3a6f79452de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.094342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ccfc8-bf9e-4d68-a173-3a6f79452de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:37.095960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ccfc8-bf9e-4d68-a173-3a6f79452de3 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-be4ccfc8-bf9e-4d68-a173-3a6f79452de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:37.116565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:37.117485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:37.118391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:37.118391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:37.118391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:37.118391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:37.118391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 396/794] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:37.119078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ccfc8-bf9e-4d68-a173-3a6f79452de3 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:37.166977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f648939-f37b-4b43-8c5e-3ea1a12a155a None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:37.204213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca605d76-1e99-40a5-9c2a-c3f7527af318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:37.205207 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca605d76-1e99-40a5-9c2a-c3f7527af318 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.205731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca605d76-1e99-40a5-9c2a-c3f7527af318 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.206222 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca605d76-1e99-40a5-9c2a-c3f7527af318 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.207250 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca605d76-1e99-40a5-9c2a-c3f7527af318 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:37.207777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca605d76-1e99-40a5-9c2a-c3f7527af318 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:37.209393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca605d76-1e99-40a5-9c2a-c3f7527af318 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:37.231098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 399/795] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 1 19:40:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:37.232255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca605d76-1e99-40a5-9c2a-c3f7527af318 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:37.245248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:37.271588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ccfc8-bf9e-4d68-a173-3a6f79452de3 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:37.316368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:37.326843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b55c9-6fda-40b8-8d8e-1d274f53d42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:37.328482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b55c9-6fda-40b8-8d8e-1d274f53d42d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.328748 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b55c9-6fda-40b8-8d8e-1d274f53d42d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.329026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b55c9-6fda-40b8-8d8e-1d274f53d42d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.344031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b55c9-6fda-40b8-8d8e-1d274f53d42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:37.351999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca605d76-1e99-40a5-9c2a-c3f7527af318 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-664b55c9-6fda-40b8-8d8e-1d274f53d42d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:37.354205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:37.355107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:37.355918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 397/796] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:37.356595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b55c9-6fda-40b8-8d8e-1d274f53d42d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:37.401206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788b61c-2892-4a14-979b-47563e71732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:37.402591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788b61c-2892-4a14-979b-47563e71732a None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.403070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788b61c-2892-4a14-979b-47563e71732a None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.403538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788b61c-2892-4a14-979b-47563e71732a None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.404507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788b61c-2892-4a14-979b-47563e71732a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:37.405049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788b61c-2892-4a14-979b-47563e71732a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:37.406803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788b61c-2892-4a14-979b-47563e71732a None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:37.419907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 400/797] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:37.421201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788b61c-2892-4a14-979b-47563e71732a None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:37.497636 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b55c9-6fda-40b8-8d8e-1d274f53d42d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:37.534038 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788b61c-2892-4a14-979b-47563e71732a None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:37.542064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6a936-bb50-4ecc-82ec-cbbf8646be86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:37.543287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6a936-bb50-4ecc-82ec-cbbf8646be86 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.543572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6a936-bb50-4ecc-82ec-cbbf8646be86 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.543846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6a936-bb50-4ecc-82ec-cbbf8646be86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.556196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6a936-bb50-4ecc-82ec-cbbf8646be86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:37.557686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6a936-bb50-4ecc-82ec-cbbf8646be86 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:37.559704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:37.571779 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:37.579274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b999d16-7138-4d61-8389-dd552b34bf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:37.579954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b999d16-7138-4d61-8389-dd552b34bf10 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.580233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b999d16-7138-4d61-8389-dd552b34bf10 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.580503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b999d16-7138-4d61-8389-dd552b34bf10 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.581258 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b999d16-7138-4d61-8389-dd552b34bf10 None admin] RBAC: Adding query filter params (name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:37.581561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b999d16-7138-4d61-8389-dd552b34bf10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:37.582967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b999d16-7138-4d61-8389-dd552b34bf10 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-37b6a936-bb50-4ecc-82ec-cbbf8646be86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:37.583684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:37.585308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:37.588029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:37.588029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:37.588029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:37.588029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:37.588029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 398/798] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:37.589188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6a936-bb50-4ecc-82ec-cbbf8646be86 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:37.596998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 401/799] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:37.596998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a985e5-a85d-4c50-9e98-2286ab271732 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.596998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a985e5-a85d-4c50-9e98-2286ab271732 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.596998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a985e5-a85d-4c50-9e98-2286ab271732 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.596998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 402/800] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:37.596998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a985e5-a85d-4c50-9e98-2286ab271732 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:37.704206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6a936-bb50-4ecc-82ec-cbbf8646be86 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:37.710630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a985e5-a85d-4c50-9e98-2286ab271732 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:37.739312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bc544-7481-4b9e-844a-478b7d4312d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:37.740416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc544-7481-4b9e-844a-478b7d4312d6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.740911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc544-7481-4b9e-844a-478b7d4312d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.741284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bc544-7481-4b9e-844a-478b7d4312d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.745481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841b1eb-a096-4a9d-8cff-8e336d83ed07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:37.746162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841b1eb-a096-4a9d-8cff-8e336d83ed07 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.746438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841b1eb-a096-4a9d-8cff-8e336d83ed07 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.746710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841b1eb-a096-4a9d-8cff-8e336d83ed07 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.747419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841b1eb-a096-4a9d-8cff-8e336d83ed07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:37.747774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841b1eb-a096-4a9d-8cff-8e336d83ed07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:37.749157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841b1eb-a096-4a9d-8cff-8e336d83ed07 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:37.749300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bc544-7481-4b9e-844a-478b7d4312d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:37.750658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bc544-7481-4b9e-844a-478b7d4312d6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:37.760728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 403/801] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:37.762557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841b1eb-a096-4a9d-8cff-8e336d83ed07 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-d29bc544-7481-4b9e-844a-478b7d4312d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:37.765558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:37.766524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:37.767462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:37.767462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:37.767462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:37.767462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:37.767462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:37.767462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:37.767462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:37.767462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:37.767462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:40:37.767462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:37.771238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 399/802] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 1 19:40:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:37.772486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a351-4912-4961-bd78-bc038eb14591 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.772774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a351-4912-4961-bd78-bc038eb14591 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.773049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0a351-4912-4961-bd78-bc038eb14591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.792323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-bfc0a351-4912-4961-bd78-bc038eb14591 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:37.793703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-bfc0a351-4912-4961-bd78-bc038eb14591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:37.825833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:37.847817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:37.873510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 400/803] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:37.873510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a351-4912-4961-bd78-bc038eb14591 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:37.879166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841b1eb-a096-4a9d-8cff-8e336d83ed07 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:37.917848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d075-ef67-40a6-a263-450a6df6170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:37.918727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d075-ef67-40a6-a263-450a6df6170e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:37.919069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d075-ef67-40a6-a263-450a6df6170e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:37.919407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4d075-ef67-40a6-a263-450a6df6170e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:37.930554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4d075-ef67-40a6-a263-450a6df6170e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:37.932073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4d075-ef67-40a6-a263-450a6df6170e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-e2c4d075-ef67-40a6-a263-450a6df6170e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:37.951949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:37.953135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:37.954143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:37.954143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:37.954143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:37.954143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:37.958470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 404/804] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:37.959098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d075-ef67-40a6-a263-450a6df6170e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:37.991797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0a351-4912-4961-bd78-bc038eb14591 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:38.028145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761df36d-713a-4a67-ab85-e74c07af81be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:38.028848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761df36d-713a-4a67-ab85-e74c07af81be None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.029129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761df36d-713a-4a67-ab85-e74c07af81be None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.029400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761df36d-713a-4a67-ab85-e74c07af81be None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.037068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761df36d-713a-4a67-ab85-e74c07af81be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:38.038534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761df36d-713a-4a67-ab85-e74c07af81be None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-761df36d-713a-4a67-ab85-e74c07af81be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:38.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:38.053533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:38.059154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:38.059154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:38.059154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:38.059154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:38.059154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:38.059154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:38.059154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:38.059154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:38.059154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 401/805] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:38.059154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761df36d-713a-4a67-ab85-e74c07af81be None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:38.089503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4d075-ef67-40a6-a263-450a6df6170e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:38.097645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:38.105466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:38.127178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302be5d-0e13-4bba-9bb8-2f0e0e441dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:38.127842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302be5d-0e13-4bba-9bb8-2f0e0e441dd2 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.128267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302be5d-0e13-4bba-9bb8-2f0e0e441dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.128546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302be5d-0e13-4bba-9bb8-2f0e0e441dd2 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.129930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302be5d-0e13-4bba-9bb8-2f0e0e441dd2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:38.130237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302be5d-0e13-4bba-9bb8-2f0e0e441dd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:38.131668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302be5d-0e13-4bba-9bb8-2f0e0e441dd2 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:38.151445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 405/806] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 1 19:40:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:38.153092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302be5d-0e13-4bba-9bb8-2f0e0e441dd2 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:38.172862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761df36d-713a-4a67-ab85-e74c07af81be None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:38.218410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119a23-9c50-4bc7-8537-fb4d19b289f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:38.219179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119a23-9c50-4bc7-8537-fb4d19b289f0 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.219522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119a23-9c50-4bc7-8537-fb4d19b289f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.219860 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119a23-9c50-4bc7-8537-fb4d19b289f0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.220721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4119a23-9c50-4bc7-8537-fb4d19b289f0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:38.223072 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4119a23-9c50-4bc7-8537-fb4d19b289f0 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-a4119a23-9c50-4bc7-8537-fb4d19b289f0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:38.239604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:38.240785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 19:40:38.241844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:38.248358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 402/807] 10.222.0.66 () {56 vars in 1129 bytes} [Fri Jul 1 19:40:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:38.248358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119a23-9c50-4bc7-8537-fb4d19b289f0 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:38.254867 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302be5d-0e13-4bba-9bb8-2f0e0e441dd2 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:38.291296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416c756-f420-4453-a10a-b428945ab6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:38.291971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416c756-f420-4453-a10a-b428945ab6e6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.292247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416c756-f420-4453-a10a-b428945ab6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.292516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416c756-f420-4453-a10a-b428945ab6e6 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.293235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416c756-f420-4453-a10a-b428945ab6e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:38.293533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416c756-f420-4453-a10a-b428945ab6e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:38.294922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416c756-f420-4453-a10a-b428945ab6e6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:38.307232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 406/808] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:38.308059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416c756-f420-4453-a10a-b428945ab6e6 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:38.343925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:38.375778 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119a23-9c50-4bc7-8537-fb4d19b289f0 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:38.385204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:38.413164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff9798-33d4-4413-a42f-6ff6c426b196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:38.414082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff9798-33d4-4413-a42f-6ff6c426b196 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.414951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff9798-33d4-4413-a42f-6ff6c426b196 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.415374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff9798-33d4-4413-a42f-6ff6c426b196 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.416294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff9798-33d4-4413-a42f-6ff6c426b196 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:38.416743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff9798-33d4-4413-a42f-6ff6c426b196 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:38.418343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff9798-33d4-4413-a42f-6ff6c426b196 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:38.427180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416c756-f420-4453-a10a-b428945ab6e6 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:38.437602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 403/809] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:38.438693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff9798-33d4-4413-a42f-6ff6c426b196 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:38.463893 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea84170-9ac8-4cb8-abd0-1c52c97d32ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:38.464600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea84170-9ac8-4cb8-abd0-1c52c97d32ef None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.464810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea84170-9ac8-4cb8-abd0-1c52c97d32ef None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.465081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea84170-9ac8-4cb8-abd0-1c52c97d32ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.540185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea84170-9ac8-4cb8-abd0-1c52c97d32ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:38.541763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea84170-9ac8-4cb8-abd0-1c52c97d32ef None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:38.548252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff9798-33d4-4413-a42f-6ff6c426b196 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:38.569718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:38.582570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfdc62-3b20-4619-a5b0-2eb9fa4eae5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:38.583216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfdc62-3b20-4619-a5b0-2eb9fa4eae5d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.583491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfdc62-3b20-4619-a5b0-2eb9fa4eae5d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.583756 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfdc62-3b20-4619-a5b0-2eb9fa4eae5d None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.584442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfdc62-3b20-4619-a5b0-2eb9fa4eae5d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:38.584743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfdc62-3b20-4619-a5b0-2eb9fa4eae5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:38.586136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfdc62-3b20-4619-a5b0-2eb9fa4eae5d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:38.590750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 404/810] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 1 19:40:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:38.591505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfdc62-3b20-4619-a5b0-2eb9fa4eae5d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:38.622061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:38.652943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 407/811] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:40:38.654023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea84170-9ac8-4cb8-abd0-1c52c97d32ef None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:38.693098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfdc62-3b20-4619-a5b0-2eb9fa4eae5d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:38.731106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698289c-e17d-47be-b9b1-9b27d3c9a4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:38.731913 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698289c-e17d-47be-b9b1-9b27d3c9a4d7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.732187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698289c-e17d-47be-b9b1-9b27d3c9a4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.732453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698289c-e17d-47be-b9b1-9b27d3c9a4d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.741875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698289c-e17d-47be-b9b1-9b27d3c9a4d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:38.743304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698289c-e17d-47be-b9b1-9b27d3c9a4d7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-6698289c-e17d-47be-b9b1-9b27d3c9a4d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:38.758300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:38.759294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 405/812] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 1 19:40:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e665685-eeee-4216-9a5a-acaf01b64c05 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.761461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e665685-eeee-4216-9a5a-acaf01b64c05 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.770007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e665685-eeee-4216-9a5a-acaf01b64c05 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.770007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 406/813] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:38.770007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e665685-eeee-4216-9a5a-acaf01b64c05 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:38.772173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea84170-9ac8-4cb8-abd0-1c52c97d32ef None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:38.797634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:38.808276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc90ee-3a3a-4ffa-9984-fbb10700893d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:38.809160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc90ee-3a3a-4ffa-9984-fbb10700893d None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.809502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc90ee-3a3a-4ffa-9984-fbb10700893d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.809798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc90ee-3a3a-4ffa-9984-fbb10700893d None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/c2bce9daec8c446d89eb6cc5950d6649/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.812193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc90ee-3a3a-4ffa-9984-fbb10700893d None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=c2bce9daec8c446d89eb6cc5950d6649, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:38.813457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc90ee-3a3a-4ffa-9984-fbb10700893d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:38.825886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 408/814] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:40:38 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/c2bce9daec8c446d89eb6cc5950d6649/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:40:38.826423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc90ee-3a3a-4ffa-9984-fbb10700893d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:38.873963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:38.888247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e665685-eeee-4216-9a5a-acaf01b64c05 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:38.939627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4a1b3-0d55-4379-958a-5e8a8d12307f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:38.940306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4a1b3-0d55-4379-958a-5e8a8d12307f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:38.940582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4a1b3-0d55-4379-958a-5e8a8d12307f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:38.940851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4a1b3-0d55-4379-958a-5e8a8d12307f None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:38.941564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4a1b3-0d55-4379-958a-5e8a8d12307f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:38.941944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4a1b3-0d55-4379-958a-5e8a8d12307f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:38.943466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4a1b3-0d55-4379-958a-5e8a8d12307f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:38.950652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 407/815] 10.222.0.66 () {56 vars in 1124 bytes} [Fri Jul 1 19:40:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:38.951517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4a1b3-0d55-4379-958a-5e8a8d12307f None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:38.965437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc90ee-3a3a-4ffa-9984-fbb10700893d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:39.009064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e7abc-f32c-46f2-80b2-3e1d6b82a723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:39.009910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e7abc-f32c-46f2-80b2-3e1d6b82a723 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.010192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e7abc-f32c-46f2-80b2-3e1d6b82a723 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.010474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e7abc-f32c-46f2-80b2-3e1d6b82a723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.021814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e7abc-f32c-46f2-80b2-3e1d6b82a723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:39.023283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e7abc-f32c-46f2-80b2-3e1d6b82a723 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-680e7abc-f32c-46f2-80b2-3e1d6b82a723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:39.042896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:39.043991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:39.045036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:39.045036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:39.045036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:39.045036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:39.045036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 409/816] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:39.047520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01519fd-f2aa-4633-b343-407241d251c5 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.048013 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01519fd-f2aa-4633-b343-407241d251c5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.048338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01519fd-f2aa-4633-b343-407241d251c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.068456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c01519fd-f2aa-4633-b343-407241d251c5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:39.069686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c01519fd-f2aa-4633-b343-407241d251c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:39.078826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4a1b3-0d55-4379-958a-5e8a8d12307f None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:39.099120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:39.120372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392baf23-bc91-433d-86de-506f308ae983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:39.121039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392baf23-bc91-433d-86de-506f308ae983 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.121322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392baf23-bc91-433d-86de-506f308ae983 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.121606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392baf23-bc91-433d-86de-506f308ae983 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.122509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392baf23-bc91-433d-86de-506f308ae983 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:39.122836 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392baf23-bc91-433d-86de-506f308ae983 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:39.124815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392baf23-bc91-433d-86de-506f308ae983 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:39.142082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 410/817] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:39.142878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01519fd-f2aa-4633-b343-407241d251c5 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:39.155251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 408/818] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 1 19:40:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:39.155251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392baf23-bc91-433d-86de-506f308ae983 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:39.176157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:39.256025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01519fd-f2aa-4633-b343-407241d251c5 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:39.276768 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392baf23-bc91-433d-86de-506f308ae983 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:39.307308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53474aaf-b31b-44d0-9321-058e23b39e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:39.307884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53474aaf-b31b-44d0-9321-058e23b39e76 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.308161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53474aaf-b31b-44d0-9321-058e23b39e76 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.308433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53474aaf-b31b-44d0-9321-058e23b39e76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.309309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53474aaf-b31b-44d0-9321-058e23b39e76 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:39.310889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53474aaf-b31b-44d0-9321-058e23b39e76 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:39.315024 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b7bd6-b76f-4200-aacd-d80580049365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:39.315781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b7bd6-b76f-4200-aacd-d80580049365 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.316156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b7bd6-b76f-4200-aacd-d80580049365 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.316522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b7bd6-b76f-4200-aacd-d80580049365 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.317357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b7bd6-b76f-4200-aacd-d80580049365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:39.317800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b7bd6-b76f-4200-aacd-d80580049365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:39.319268 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b7bd6-b76f-4200-aacd-d80580049365 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:39.335022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 411/819] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 1 19:40:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:39.336096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53474aaf-b31b-44d0-9321-058e23b39e76 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:39.336712 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 409/820] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:39.337729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b7bd6-b76f-4200-aacd-d80580049365 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:39.372027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:39.460344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:39.505776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b7bd6-b76f-4200-aacd-d80580049365 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:39.506368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53474aaf-b31b-44d0-9321-058e23b39e76 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:39.553173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d337b4-fb38-443c-a9ef-86559776d884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:39.553865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d337b4-fb38-443c-a9ef-86559776d884 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.554212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d337b4-fb38-443c-a9ef-86559776d884 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.554482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d337b4-fb38-443c-a9ef-86559776d884 None admin] PATH_INFO: `/v3/services/network` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.555184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d337b4-fb38-443c-a9ef-86559776d884 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:39.556561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d337b4-fb38-443c-a9ef-86559776d884 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:39.561284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c16eb-e1a7-4a3f-91c6-45844d7d1a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:39.562187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c16eb-e1a7-4a3f-91c6-45844d7d1a66 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.562471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c16eb-e1a7-4a3f-91c6-45844d7d1a66 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.562746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c16eb-e1a7-4a3f-91c6-45844d7d1a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-26d337b4-fb38-443c-a9ef-86559776d884 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:39.573412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:39.574454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:39.575277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 410/821] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:39 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:39.576095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d337b4-fb38-443c-a9ef-86559776d884 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:39.576162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c16eb-e1a7-4a3f-91c6-45844d7d1a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:39.576162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c16eb-e1a7-4a3f-91c6-45844d7d1a66 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6e3c16eb-e1a7-4a3f-91c6-45844d7d1a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:39.598534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:39.599502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:39.600372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:39.600372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:39.600372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:39.600372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:39.600631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 412/822] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:39.601371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c16eb-e1a7-4a3f-91c6-45844d7d1a66 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:39.688864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:39.704065 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d337b4-fb38-443c-a9ef-86559776d884 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:39.723436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c16eb-e1a7-4a3f-91c6-45844d7d1a66 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:39.748818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43f1dc-06db-4018-9c92-1cfbd3cec807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:39.750397 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43f1dc-06db-4018-9c92-1cfbd3cec807 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.750676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43f1dc-06db-4018-9c92-1cfbd3cec807 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.751279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43f1dc-06db-4018-9c92-1cfbd3cec807 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.760442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43f1dc-06db-4018-9c92-1cfbd3cec807 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:39.762138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d232d-7937-41ae-933d-81f2d11fbda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:39.762324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43f1dc-06db-4018-9c92-1cfbd3cec807 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:39.762936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d232d-7937-41ae-933d-81f2d11fbda4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.763284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d232d-7937-41ae-933d-81f2d11fbda4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.763666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d232d-7937-41ae-933d-81f2d11fbda4 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.763966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:39.765227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d232d-7937-41ae-933d-81f2d11fbda4 None admin] RBAC: Adding query filter params (name=network) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:39.766357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d232d-7937-41ae-933d-81f2d11fbda4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:39.767725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295d232d-7937-41ae-933d-81f2d11fbda4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:39.776521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 413/823] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:40:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:39.777404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d232d-7937-41ae-933d-81f2d11fbda4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-cc43f1dc-06db-4018-9c92-1cfbd3cec807 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:39.783607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:39.784502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:39.786045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:39.786045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:39.786045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:39.786045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:39.786045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:39.786045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:39.786045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:39.786045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:39.789634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 411/824] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:39.793760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43f1dc-06db-4018-9c92-1cfbd3cec807 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:39.879646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d232d-7937-41ae-933d-81f2d11fbda4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:39.902357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43f1dc-06db-4018-9c92-1cfbd3cec807 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:39.915076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb970aa3-9fa9-4a5a-a3f2-7b96cde71ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:39.915745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb970aa3-9fa9-4a5a-a3f2-7b96cde71ea4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.916025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb970aa3-9fa9-4a5a-a3f2-7b96cde71ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.916301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb970aa3-9fa9-4a5a-a3f2-7b96cde71ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.917021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb970aa3-9fa9-4a5a-a3f2-7b96cde71ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:39.917322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb970aa3-9fa9-4a5a-a3f2-7b96cde71ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:39.918730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb970aa3-9fa9-4a5a-a3f2-7b96cde71ea4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:39.921911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:39.936313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 414/825] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:39.938245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb970aa3-9fa9-4a5a-a3f2-7b96cde71ea4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:39.942751 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0bc12-aa48-400d-9c29-b94c399304a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:39.943538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0bc12-aa48-400d-9c29-b94c399304a6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:39.943923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0bc12-aa48-400d-9c29-b94c399304a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:39.944301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0bc12-aa48-400d-9c29-b94c399304a6 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:39.945121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0bc12-aa48-400d-9c29-b94c399304a6 None admin] RBAC: Adding query filter params (type=network) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:39.945533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0bc12-aa48-400d-9c29-b94c399304a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:39.947282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0bc12-aa48-400d-9c29-b94c399304a6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:39.953932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 412/826] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:40:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:39.954838 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0bc12-aa48-400d-9c29-b94c399304a6 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:40.020174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:40.050850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb970aa3-9fa9-4a5a-a3f2-7b96cde71ea4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:40.062007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0bc12-aa48-400d-9c29-b94c399304a6 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:40.091673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ab26d-eaf2-4b7d-a6bd-82fca9b7c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:40.093025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ab26d-eaf2-4b7d-a6bd-82fca9b7c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:40.094263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ab26d-eaf2-4b7d-a6bd-82fca9b7c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:40.094733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ab26d-eaf2-4b7d-a6bd-82fca9b7c6b3 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:40.095639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ab26d-eaf2-4b7d-a6bd-82fca9b7c6b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:40.096231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ab26d-eaf2-4b7d-a6bd-82fca9b7c6b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:40.098515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ab26d-eaf2-4b7d-a6bd-82fca9b7c6b3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:40.102222 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fb305-1621-4250-82b7-5470f4d171a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:40.103209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb305-1621-4250-82b7-5470f4d171a2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:40.103633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb305-1621-4250-82b7-5470f4d171a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:40.104045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb305-1621-4250-82b7-5470f4d171a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:40.111672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fb305-1621-4250-82b7-5470f4d171a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:40.113403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fb305-1621-4250-82b7-5470f4d171a2 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:40.118142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 415/827] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:40.119527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ab26d-eaf2-4b7d-a6bd-82fca9b7c6b3 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-ac1fb305-1621-4250-82b7-5470f4d171a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:40.131954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:40.132843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:40.133843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:40.133843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:40.133843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:40.133843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:40.133843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:40.133843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:40.133843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:40.133843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:40.140259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 413/828] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:40.140259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fb305-1621-4250-82b7-5470f4d171a2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:40.196737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:40.253858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ab26d-eaf2-4b7d-a6bd-82fca9b7c6b3 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:40.274156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fb305-1621-4250-82b7-5470f4d171a2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:40.299783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225e536-4cd3-497c-9fa8-cd489f423ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:40.300632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225e536-4cd3-497c-9fa8-cd489f423ddb None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:40.301023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225e536-4cd3-497c-9fa8-cd489f423ddb None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:40.301450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225e536-4cd3-497c-9fa8-cd489f423ddb None admin] PATH_INFO: `/v3/endpoints` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:40.302860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225e536-4cd3-497c-9fa8-cd489f423ddb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:40.304838 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:40.306290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225e536-4cd3-497c-9fa8-cd489f423ddb None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:40.321201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5a5fc-2dc4-4591-9680-155a3251ee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:40.321944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5a5fc-2dc4-4591-9680-155a3251ee11 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:40.322228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5a5fc-2dc4-4591-9680-155a3251ee11 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:40.322501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5a5fc-2dc4-4591-9680-155a3251ee11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:40.323332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5a5fc-2dc4-4591-9680-155a3251ee11 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=ad06f31859f9413f9db6039ac2abf982) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:40.323755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5a5fc-2dc4-4591-9680-155a3251ee11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:40.325326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5a5fc-2dc4-4591-9680-155a3251ee11 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:40.337793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 414/830] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:40 2022] GET /identity/v3/role_assignments?user.id=ad06f31859f9413f9db6039ac2abf982&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:40.337793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5a5fc-2dc4-4591-9680-155a3251ee11 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:40.338098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 416/829] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 1 19:40:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:40.501036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5a5fc-2dc4-4591-9680-155a3251ee11 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:40.553460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123885a2-adc1-4081-948d-9f1307cf65b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:40.554169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123885a2-adc1-4081-948d-9f1307cf65b3 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:40.554451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123885a2-adc1-4081-948d-9f1307cf65b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:40.554728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123885a2-adc1-4081-948d-9f1307cf65b3 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:40.555526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123885a2-adc1-4081-948d-9f1307cf65b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:40.555832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123885a2-adc1-4081-948d-9f1307cf65b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:40.557265 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123885a2-adc1-4081-948d-9f1307cf65b3 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:40.571623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 415/831] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:40.577934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225e536-4cd3-497c-9fa8-cd489f423ddb None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:40.699345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225e536-4cd3-497c-9fa8-cd489f423ddb None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:40.710841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:40.737618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649d1f0-49fe-45ea-8882-7018e50d58a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:40.738540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649d1f0-49fe-45ea-8882-7018e50d58a7 None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:40.738822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649d1f0-49fe-45ea-8882-7018e50d58a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:40.739097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649d1f0-49fe-45ea-8882-7018e50d58a7 None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/cfa25a63dd6f4ad08b0f858356528284/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:40.756167 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649d1f0-49fe-45ea-8882-7018e50d58a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=cfa25a63dd6f4ad08b0f858356528284, role_id=0db2f13c4dbe42a4a0d8ffa0bb5633fd)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:40.757536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649d1f0-49fe-45ea-8882-7018e50d58a7 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:40.773975 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 417/832] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:40:40 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/cfa25a63dd6f4ad08b0f858356528284/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:40:41.169144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd19c2-9990-41a9-811f-b140d86cad19 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:41.169610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd19c2-9990-41a9-811f-b140d86cad19 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:41.169935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd19c2-9990-41a9-811f-b140d86cad19 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:41.172996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 416/833] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:41.179567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e4cbd-e68e-43ab-8d6a-f7030e732d7d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:41.179850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e4cbd-e68e-43ab-8d6a-f7030e732d7d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:41.180121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e4cbd-e68e-43ab-8d6a-f7030e732d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:41.197166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d76e4cbd-e68e-43ab-8d6a-f7030e732d7d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:41.198482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d76e4cbd-e68e-43ab-8d6a-f7030e732d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:41.272285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 418/834] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:41.302424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd19c2-9990-41a9-811f-b140d86cad19 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:41.351334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:41.413887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd19c2-9990-41a9-811f-b140d86cad19 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:41.451942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad804fb-6d9a-4c4c-a9d0-6b1b6d348521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:41.453310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad804fb-6d9a-4c4c-a9d0-6b1b6d348521 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:41.453605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad804fb-6d9a-4c4c-a9d0-6b1b6d348521 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:41.453882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad804fb-6d9a-4c4c-a9d0-6b1b6d348521 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:41.465014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad804fb-6d9a-4c4c-a9d0-6b1b6d348521 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-9ad804fb-6d9a-4c4c-a9d0-6b1b6d348521 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:41.473212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:41.474583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:41.476026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 417/835] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:41.481150 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e4cbd-e68e-43ab-8d6a-f7030e732d7d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:41.586015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:41.595246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e4cbd-e68e-43ab-8d6a-f7030e732d7d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:41.631101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7702da3-a7f8-420e-a942-cddb80f2a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:41.631772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7702da3-a7f8-420e-a942-cddb80f2a00f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:41.632052 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7702da3-a7f8-420e-a942-cddb80f2a00f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:41.632326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7702da3-a7f8-420e-a942-cddb80f2a00f None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:41.633080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7702da3-a7f8-420e-a942-cddb80f2a00f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:41.633382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7702da3-a7f8-420e-a942-cddb80f2a00f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:41.634956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7702da3-a7f8-420e-a942-cddb80f2a00f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:41.644106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 419/836] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:41.649293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad804fb-6d9a-4c4c-a9d0-6b1b6d348521 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:41.657117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09756f7-4f15-474d-8aa0-c436602d2963 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:41.657398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09756f7-4f15-474d-8aa0-c436602d2963 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:41.657708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09756f7-4f15-474d-8aa0-c436602d2963 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:41.663161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 420/837] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:41.664761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddccfd-58a2-4fd0-b110-cede09dadb39 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:41.665053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddccfd-58a2-4fd0-b110-cede09dadb39 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:41.665330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddccfd-58a2-4fd0-b110-cede09dadb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:41.683635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-b7ddccfd-58a2-4fd0-b110-cede09dadb39 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:41.684831 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b7ddccfd-58a2-4fd0-b110-cede09dadb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:41.777799 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 421/838] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:41.783393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad804fb-6d9a-4c4c-a9d0-6b1b6d348521 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:41.784990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:41.794954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddccfd-58a2-4fd0-b110-cede09dadb39 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:41.827937 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c679e25-f68a-4b22-95cd-7f47a2776c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:41.828996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c679e25-f68a-4b22-95cd-7f47a2776c12 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:41.829301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c679e25-f68a-4b22-95cd-7f47a2776c12 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:41.829983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c679e25-f68a-4b22-95cd-7f47a2776c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:41.844198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c679e25-f68a-4b22-95cd-7f47a2776c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:41.845686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c679e25-f68a-4b22-95cd-7f47a2776c12 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-0c679e25-f68a-4b22-95cd-7f47a2776c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:41.864256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:41.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:41.866127 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:41.866127 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:41.866127 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:41.866127 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:41.866127 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 418/839] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:41.875126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c679e25-f68a-4b22-95cd-7f47a2776c12 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:41.893014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:41.921602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddccfd-58a2-4fd0-b110-cede09dadb39 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:41.955659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644263d-56b3-475a-9082-e7a4a5bb4cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:41.956592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644263d-56b3-475a-9082-e7a4a5bb4cb8 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:41.956879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644263d-56b3-475a-9082-e7a4a5bb4cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:41.957156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644263d-56b3-475a-9082-e7a4a5bb4cb8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:41.957861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644263d-56b3-475a-9082-e7a4a5bb4cb8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:41.959398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644263d-56b3-475a-9082-e7a4a5bb4cb8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-d644263d-56b3-475a-9082-e7a4a5bb4cb8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:41.973916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:41.974953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 19:40:41.975766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:41.980031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 422/840] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:40:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:41.981297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644263d-56b3-475a-9082-e7a4a5bb4cb8 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:42.010612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c679e25-f68a-4b22-95cd-7f47a2776c12 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:42.047180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fc5de-1319-4554-8da1-65a6a2399753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:42.047762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fc5de-1319-4554-8da1-65a6a2399753 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:42.048092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fc5de-1319-4554-8da1-65a6a2399753 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:42.048375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fc5de-1319-4554-8da1-65a6a2399753 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:42.049106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fc5de-1319-4554-8da1-65a6a2399753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:42.049413 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fc5de-1319-4554-8da1-65a6a2399753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:42.050820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fc5de-1319-4554-8da1-65a6a2399753 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:42.063671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 419/841] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:42.068770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fc5de-1319-4554-8da1-65a6a2399753 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:42.089567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:42.111002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644263d-56b3-475a-9082-e7a4a5bb4cb8 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:42.147943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa339-e302-4352-9fd9-29149fc6577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:42.148605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa339-e302-4352-9fd9-29149fc6577e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:42.148889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa339-e302-4352-9fd9-29149fc6577e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:42.149396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa339-e302-4352-9fd9-29149fc6577e None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:42.149910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faa339-e302-4352-9fd9-29149fc6577e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:42.150214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faa339-e302-4352-9fd9-29149fc6577e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:42.152200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faa339-e302-4352-9fd9-29149fc6577e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:42.163228 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 423/842] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:40:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:42.164886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa339-e302-4352-9fd9-29149fc6577e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:42.178765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:42.211262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fc5de-1319-4554-8da1-65a6a2399753 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:42.258279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461bdb3e-3514-4511-9f6c-45e3b2ac92d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:42.259973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bdb3e-3514-4511-9f6c-45e3b2ac92d5 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:42.260740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bdb3e-3514-4511-9f6c-45e3b2ac92d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:42.261218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461bdb3e-3514-4511-9f6c-45e3b2ac92d5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:42.272729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461bdb3e-3514-4511-9f6c-45e3b2ac92d5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:42.272898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa339-e302-4352-9fd9-29149fc6577e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:42.274873 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461bdb3e-3514-4511-9f6c-45e3b2ac92d5 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-461bdb3e-3514-4511-9f6c-45e3b2ac92d5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:42.294817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:42.296510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:42.297490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:42.297490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:42.297490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:42.297490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:42.297490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:42.297490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:42.297490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:42.297490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:42.297490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 19:40:42.297490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:42.303589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 420/843] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:40:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:42.305694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461bdb3e-3514-4511-9f6c-45e3b2ac92d5 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:42.325071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b7917-d4be-4e07-af21-9f5dd6ca3fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:42.326268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b7917-d4be-4e07-af21-9f5dd6ca3fa1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:42.327129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b7917-d4be-4e07-af21-9f5dd6ca3fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:42.327608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b7917-d4be-4e07-af21-9f5dd6ca3fa1 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:42.328513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b7917-d4be-4e07-af21-9f5dd6ca3fa1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:42.328995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b7917-d4be-4e07-af21-9f5dd6ca3fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:42.331151 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b7917-d4be-4e07-af21-9f5dd6ca3fa1 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:42.340499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 424/844] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:40:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:42.404000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:42.433894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461bdb3e-3514-4511-9f6c-45e3b2ac92d5 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:42.472392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c12a07-ece9-4bf8-8bca-67e5b652a3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:42.473396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c12a07-ece9-4bf8-8bca-67e5b652a3ec None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:42.473895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c12a07-ece9-4bf8-8bca-67e5b652a3ec None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:42.474344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c12a07-ece9-4bf8-8bca-67e5b652a3ec None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:42.475417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c12a07-ece9-4bf8-8bca-67e5b652a3ec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:42.475911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c12a07-ece9-4bf8-8bca-67e5b652a3ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:42.477543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c12a07-ece9-4bf8-8bca-67e5b652a3ec None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:42.498200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 421/845] 10.222.0.66 () {56 vars in 1135 bytes} [Fri Jul 1 19:40:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:42.508288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b7917-d4be-4e07-af21-9f5dd6ca3fa1 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:42.590131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:42.617668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b7917-d4be-4e07-af21-9f5dd6ca3fa1 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:42.652278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5ad1a-caf0-4f48-890f-d46bb349b1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:42.652939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5ad1a-caf0-4f48-890f-d46bb349b1aa None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:42.653249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5ad1a-caf0-4f48-890f-d46bb349b1aa None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:42.653485 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5ad1a-caf0-4f48-890f-d46bb349b1aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:42.706786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248435f-7002-448d-8e5a-d8368f29e71a None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:42.707129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248435f-7002-448d-8e5a-d8368f29e71a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:42.707643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248435f-7002-448d-8e5a-d8368f29e71a None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:42.709312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 422/846] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:42.715372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db8761-2f9c-49ea-9021-483ccfb6f9a8 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:42.715654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db8761-2f9c-49ea-9021-483ccfb6f9a8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:42.715929 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db8761-2f9c-49ea-9021-483ccfb6f9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:42.729725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5ad1a-caf0-4f48-890f-d46bb349b1aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:42.731183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5ad1a-caf0-4f48-890f-d46bb349b1aa None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:42.737752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-40db8761-2f9c-49ea-9021-483ccfb6f9a8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:42.739269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-40db8761-2f9c-49ea-9021-483ccfb6f9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:42.813955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:42.829635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 423/847] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:42.846536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 425/848] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:40:42.851947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db8761-2f9c-49ea-9021-483ccfb6f9a8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:42.857732 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5ad1a-caf0-4f48-890f-d46bb349b1aa None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:42.917419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:42.961939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db8761-2f9c-49ea-9021-483ccfb6f9a8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:42.990206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5ad1a-caf0-4f48-890f-d46bb349b1aa None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:42.999788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df19f09-78d9-4e96-9946-680cfa07a328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.000643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df19f09-78d9-4e96-9946-680cfa07a328 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.000995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df19f09-78d9-4e96-9946-680cfa07a328 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.001336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df19f09-78d9-4e96-9946-680cfa07a328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.012516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df19f09-78d9-4e96-9946-680cfa07a328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:43.014169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df19f09-78d9-4e96-9946-680cfa07a328 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:43.025904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae132826-5058-485b-a4b3-8b5fc50436fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.026721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae132826-5058-485b-a4b3-8b5fc50436fd None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.027001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae132826-5058-485b-a4b3-8b5fc50436fd None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.027275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae132826-5058-485b-a4b3-8b5fc50436fd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.027956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae132826-5058-485b-a4b3-8b5fc50436fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:43.029318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae132826-5058-485b-a4b3-8b5fc50436fd None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-1df19f09-78d9-4e96-9946-680cfa07a328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:43.032328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:43.033158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:43.034063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:43.034063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:43.034063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:43.034063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:43.034063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 424/849] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:43.034971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660627a8-ee3c-46b1-8852-d1dbcc066dec None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.035070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660627a8-ee3c-46b1-8852-d1dbcc066dec None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.035560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660627a8-ee3c-46b1-8852-d1dbcc066dec None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.040942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 425/850] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:43.040942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660627a8-ee3c-46b1-8852-d1dbcc066dec None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-ae132826-5058-485b-a4b3-8b5fc50436fd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:43.050583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:43.051494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:43.053545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 426/851] 10.222.0.66 () {56 vars in 1132 bytes} [Fri Jul 1 19:40:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:43.055777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3db63-c07f-4fc4-a868-8c7703146f24 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.055777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3db63-c07f-4fc4-a868-8c7703146f24 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.055777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3db63-c07f-4fc4-a868-8c7703146f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.070907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f4e3db63-c07f-4fc4-a868-8c7703146f24 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:43.072021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f4e3db63-c07f-4fc4-a868-8c7703146f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:43.102040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:43.141991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 427/852] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:43.143162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3db63-c07f-4fc4-a868-8c7703146f24 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:43.162678 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660627a8-ee3c-46b1-8852-d1dbcc066dec None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:43.178339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:43.201146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624d0a4-afce-42e1-a0fe-9114fcc2ad23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.202115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624d0a4-afce-42e1-a0fe-9114fcc2ad23 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.202543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624d0a4-afce-42e1-a0fe-9114fcc2ad23 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.203016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624d0a4-afce-42e1-a0fe-9114fcc2ad23 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.204486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624d0a4-afce-42e1-a0fe-9114fcc2ad23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:43.204934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624d0a4-afce-42e1-a0fe-9114fcc2ad23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:43.206594 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624d0a4-afce-42e1-a0fe-9114fcc2ad23 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:43.218802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 426/853] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:43.219760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624d0a4-afce-42e1-a0fe-9114fcc2ad23 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:43.268263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3db63-c07f-4fc4-a868-8c7703146f24 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:43.307370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4370b9a-2000-408b-81aa-7d4cafb7cc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.308014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370b9a-2000-408b-81aa-7d4cafb7cc81 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.308515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370b9a-2000-408b-81aa-7d4cafb7cc81 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.308685 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370b9a-2000-408b-81aa-7d4cafb7cc81 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.309385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4370b9a-2000-408b-81aa-7d4cafb7cc81 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:43.309719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4370b9a-2000-408b-81aa-7d4cafb7cc81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:43.311752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4370b9a-2000-408b-81aa-7d4cafb7cc81 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:43.321472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 428/854] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:40:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:43.321472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4370b9a-2000-408b-81aa-7d4cafb7cc81 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:43.330583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624d0a4-afce-42e1-a0fe-9114fcc2ad23 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:43.353173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:43.364764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b40f3-046f-4212-a45a-92dafb9c5cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.365675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b40f3-046f-4212-a45a-92dafb9c5cb8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.366042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b40f3-046f-4212-a45a-92dafb9c5cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.366379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b40f3-046f-4212-a45a-92dafb9c5cb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.377363 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b40f3-046f-4212-a45a-92dafb9c5cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-6f2b40f3-046f-4212-a45a-92dafb9c5cb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:43.385309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:43.386305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:43.387149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 427/855] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:40:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:43.387848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b40f3-046f-4212-a45a-92dafb9c5cb8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:43.425988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:43.437051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4370b9a-2000-408b-81aa-7d4cafb7cc81 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:43.472115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de681d89-0bb3-44fb-8f41-bad807c8e22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.473528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de681d89-0bb3-44fb-8f41-bad807c8e22d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.473909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de681d89-0bb3-44fb-8f41-bad807c8e22d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.474255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de681d89-0bb3-44fb-8f41-bad807c8e22d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.481925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de681d89-0bb3-44fb-8f41-bad807c8e22d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:43.484244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de681d89-0bb3-44fb-8f41-bad807c8e22d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:43.497167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b40f3-046f-4212-a45a-92dafb9c5cb8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-de681d89-0bb3-44fb-8f41-bad807c8e22d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:43.498191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:43.499195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:43.500103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:43.500103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:43.500103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:43.500103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:43.500103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:43.500103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:43.500103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:43.500103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:43.504383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 429/856] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:43.505047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de681d89-0bb3-44fb-8f41-bad807c8e22d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:43.541266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234c1e3-daba-4182-b1c4-d25e839844cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.542035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c1e3-daba-4182-b1c4-d25e839844cf None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.542314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c1e3-daba-4182-b1c4-d25e839844cf None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.542586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c1e3-daba-4182-b1c4-d25e839844cf None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.543357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234c1e3-daba-4182-b1c4-d25e839844cf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:43.543702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234c1e3-daba-4182-b1c4-d25e839844cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:43.545303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234c1e3-daba-4182-b1c4-d25e839844cf None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:43.562360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 428/857] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:40:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:43.563756 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234c1e3-daba-4182-b1c4-d25e839844cf None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:43.631381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:43.642846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de681d89-0bb3-44fb-8f41-bad807c8e22d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:43.685409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb05247-1f00-49f0-b0bc-4736a6a67e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.686963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb05247-1f00-49f0-b0bc-4736a6a67e7d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.687397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb05247-1f00-49f0-b0bc-4736a6a67e7d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.687840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb05247-1f00-49f0-b0bc-4736a6a67e7d None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.688755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb05247-1f00-49f0-b0bc-4736a6a67e7d None admin] RBAC: Adding query filter params (name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:43.689224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb05247-1f00-49f0-b0bc-4736a6a67e7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:43.690812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb05247-1f00-49f0-b0bc-4736a6a67e7d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:43.700623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 430/858] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:40:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:43.707130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb05247-1f00-49f0-b0bc-4736a6a67e7d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:43.711757 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:43.715847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234c1e3-daba-4182-b1c4-d25e839844cf None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:43.754330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0e76f-4563-4ee3-8f3e-8add65a8385b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.755084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0e76f-4563-4ee3-8f3e-8add65a8385b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.755378 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0e76f-4563-4ee3-8f3e-8add65a8385b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.755654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0e76f-4563-4ee3-8f3e-8add65a8385b None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.756438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0e76f-4563-4ee3-8f3e-8add65a8385b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:43.756742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0e76f-4563-4ee3-8f3e-8add65a8385b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:43.758241 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0e76f-4563-4ee3-8f3e-8add65a8385b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:43.772434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 429/859] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:43.778561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0e76f-4563-4ee3-8f3e-8add65a8385b None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:43.838128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb05247-1f00-49f0-b0bc-4736a6a67e7d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:43.885252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568c0d8-57f6-4787-860f-0b53b911da46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.886024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568c0d8-57f6-4787-860f-0b53b911da46 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.886301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568c0d8-57f6-4787-860f-0b53b911da46 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.886580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568c0d8-57f6-4787-860f-0b53b911da46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.901113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0e76f-4563-4ee3-8f3e-8add65a8385b None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:43.902411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568c0d8-57f6-4787-860f-0b53b911da46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:43.905205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568c0d8-57f6-4787-860f-0b53b911da46 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-8568c0d8-57f6-4787-860f-0b53b911da46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:43.927886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:43.928857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:43.929761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:43.929761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:43.929761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:43.929761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:43.933701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 431/860] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:43.935484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568c0d8-57f6-4787-860f-0b53b911da46 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:43.948247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6987bf0-b127-411a-a790-7e2da8217bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:43.948899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6987bf0-b127-411a-a790-7e2da8217bc9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:43.949177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6987bf0-b127-411a-a790-7e2da8217bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:43.949487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6987bf0-b127-411a-a790-7e2da8217bc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:43.949965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:43.950355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6987bf0-b127-411a-a790-7e2da8217bc9 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=c2bce9daec8c446d89eb6cc5950d6649) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:43.951092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6987bf0-b127-411a-a790-7e2da8217bc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:43.953321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6987bf0-b127-411a-a790-7e2da8217bc9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:43.960625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 430/861] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:43 2022] GET /identity/v3/role_assignments?user.id=c2bce9daec8c446d89eb6cc5950d6649&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:44.062703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568c0d8-57f6-4787-860f-0b53b911da46 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:44.101832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f860c9c-3537-48fa-bf42-3c6ff04a3e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:44.102829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f860c9c-3537-48fa-bf42-3c6ff04a3e91 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:44.103411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f860c9c-3537-48fa-bf42-3c6ff04a3e91 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:44.103868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f860c9c-3537-48fa-bf42-3c6ff04a3e91 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:44.104791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f860c9c-3537-48fa-bf42-3c6ff04a3e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:44.105285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f860c9c-3537-48fa-bf42-3c6ff04a3e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:44.106937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f860c9c-3537-48fa-bf42-3c6ff04a3e91 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:44.121227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 432/862] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:44.127930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6987bf0-b127-411a-a790-7e2da8217bc9 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:44.159416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:44.245262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6987bf0-b127-411a-a790-7e2da8217bc9 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:44.281853 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7ac42-32ed-4f3c-afc1-0a56c1743df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:44.283005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7ac42-32ed-4f3c-afc1-0a56c1743df2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:44.283468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7ac42-32ed-4f3c-afc1-0a56c1743df2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:44.283919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7ac42-32ed-4f3c-afc1-0a56c1743df2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:44.292522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7ac42-32ed-4f3c-afc1-0a56c1743df2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:44.294324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7ac42-32ed-4f3c-afc1-0a56c1743df2 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-5cd7ac42-32ed-4f3c-afc1-0a56c1743df2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:44.310440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:44.312054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:44.312995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:44.312995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:44.312995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:44.312995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:44.312995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:44.312995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:44.312995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:44.312995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:44.312995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:44.312995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:44.315649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 431/863] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 1 19:40:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:44.321405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f860c9c-3537-48fa-bf42-3c6ff04a3e91 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:44.393434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:44.431526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f860c9c-3537-48fa-bf42-3c6ff04a3e91 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:44.466470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0d5e9-2484-4d64-92fe-47a2a59bc0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:44.467496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0d5e9-2484-4d64-92fe-47a2a59bc0e6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:44.468487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0d5e9-2484-4d64-92fe-47a2a59bc0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:44.468941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0d5e9-2484-4d64-92fe-47a2a59bc0e6 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:44.470064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0d5e9-2484-4d64-92fe-47a2a59bc0e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:44.470586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0d5e9-2484-4d64-92fe-47a2a59bc0e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:44.472193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0d5e9-2484-4d64-92fe-47a2a59bc0e6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:44.490718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 433/864] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 1 19:40:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:44.496527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7ac42-32ed-4f3c-afc1-0a56c1743df2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:44.499567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b6f24-7290-4627-8602-a180183ef29a None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:44.499861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b6f24-7290-4627-8602-a180183ef29a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:44.500390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b6f24-7290-4627-8602-a180183ef29a None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:44.502854 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 434/865] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:44.507925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddd664-0dd3-4137-863a-02a719ac0451 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:44.508211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddd664-0dd3-4137-863a-02a719ac0451 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:44.508552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddd664-0dd3-4137-863a-02a719ac0451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:44.529937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f2ddd664-0dd3-4137-863a-02a719ac0451 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:44.532058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f2ddd664-0dd3-4137-863a-02a719ac0451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:44.591701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:44.616670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7ac42-32ed-4f3c-afc1-0a56c1743df2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:44.622908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 435/866] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:44.647834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddd664-0dd3-4137-863a-02a719ac0451 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:44.665257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c643bd-8cb6-44ba-8f99-b1f2b825a365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:44.666741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c643bd-8cb6-44ba-8f99-b1f2b825a365 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:44.667538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c643bd-8cb6-44ba-8f99-b1f2b825a365 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:44.668044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c643bd-8cb6-44ba-8f99-b1f2b825a365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:44.728943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:44.761959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c643bd-8cb6-44ba-8f99-b1f2b825a365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:44.765235 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c643bd-8cb6-44ba-8f99-b1f2b825a365 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:44.782919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddd664-0dd3-4137-863a-02a719ac0451 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:44.829666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708861c9-114a-4ba2-836d-1098f4aabbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:44.830952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708861c9-114a-4ba2-836d-1098f4aabbc5 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:44.831961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708861c9-114a-4ba2-836d-1098f4aabbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:44.832719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708861c9-114a-4ba2-836d-1098f4aabbc5 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:44.833921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708861c9-114a-4ba2-836d-1098f4aabbc5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:44.835930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708861c9-114a-4ba2-836d-1098f4aabbc5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:44.859479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 436/867] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:40:44 2022] POST /identity/v3/services => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:44.881859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:44.908224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 432/868] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:44 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:40:44.914463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708861c9-114a-4ba2-836d-1098f4aabbc5 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:45.036003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708861c9-114a-4ba2-836d-1098f4aabbc5 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:45.047983 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:45.074984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66757622-6e95-4e0d-9669-4d455392bff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:45.075840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66757622-6e95-4e0d-9669-4d455392bff3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:45.076123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66757622-6e95-4e0d-9669-4d455392bff3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:45.076395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66757622-6e95-4e0d-9669-4d455392bff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:45.087795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66757622-6e95-4e0d-9669-4d455392bff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:45.089262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66757622-6e95-4e0d-9669-4d455392bff3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-66757622-6e95-4e0d-9669-4d455392bff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:45.111050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:45.112007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:45.112898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:45.112898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:45.112898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:45.112898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:45.115165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 437/869] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:45.119954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c643bd-8cb6-44ba-8f99-b1f2b825a365 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:45.232940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c643bd-8cb6-44ba-8f99-b1f2b825a365 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:45.272551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cbbb6-1fb9-4e99-9d63-1c6bf46a9dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:45.273262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cbbb6-1fb9-4e99-9d63-1c6bf46a9dab None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:45.273541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cbbb6-1fb9-4e99-9d63-1c6bf46a9dab None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:45.273905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cbbb6-1fb9-4e99-9d63-1c6bf46a9dab None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:45.274625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cbbb6-1fb9-4e99-9d63-1c6bf46a9dab None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:45.274973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cbbb6-1fb9-4e99-9d63-1c6bf46a9dab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:45.276343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cbbb6-1fb9-4e99-9d63-1c6bf46a9dab None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:45.289967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 433/870] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:45.294957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66757622-6e95-4e0d-9669-4d455392bff3 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:45.401298 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66757622-6e95-4e0d-9669-4d455392bff3 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:45.437270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c30af-e7a0-47a7-855a-b8aa8b867767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:45.438104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c30af-e7a0-47a7-855a-b8aa8b867767 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:45.438382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c30af-e7a0-47a7-855a-b8aa8b867767 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:45.438653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c30af-e7a0-47a7-855a-b8aa8b867767 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:45.445595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c30af-e7a0-47a7-855a-b8aa8b867767 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:45.447038 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c30af-e7a0-47a7-855a-b8aa8b867767 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-0b5c30af-e7a0-47a7-855a-b8aa8b867767 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:45.461504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:45.462393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:45.463270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:45.463270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:45.463270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:45.463270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:45.463270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:45.463270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:45.463270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:45.463270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:45.463270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 438/871] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:45.469050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cbbb6-1fb9-4e99-9d63-1c6bf46a9dab None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:45.520278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:45.584956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cbbb6-1fb9-4e99-9d63-1c6bf46a9dab None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:45.621738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a42f8-4345-440a-8441-49b7c963dc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:45.622398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a42f8-4345-440a-8441-49b7c963dc1b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:45.622673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a42f8-4345-440a-8441-49b7c963dc1b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:45.622945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a42f8-4345-440a-8441-49b7c963dc1b None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:45.623820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a42f8-4345-440a-8441-49b7c963dc1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:45.624735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a42f8-4345-440a-8441-49b7c963dc1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:45.626196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a42f8-4345-440a-8441-49b7c963dc1b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:45.638979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 434/872] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:45.644886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c30af-e7a0-47a7-855a-b8aa8b867767 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:45.673002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:45.746216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb6e58-9919-452f-825c-f376ca28bcc7 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:45.746440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb6e58-9919-452f-825c-f376ca28bcc7 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:45.746764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb6e58-9919-452f-825c-f376ca28bcc7 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:45.749281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 435/873] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:45.753295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c51401-3cb2-4d56-969e-9315871b6978 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:45.753598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c51401-3cb2-4d56-969e-9315871b6978 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:45.753879 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c51401-3cb2-4d56-969e-9315871b6978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:45.754379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c30af-e7a0-47a7-855a-b8aa8b867767 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:45.771988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-45c51401-3cb2-4d56-969e-9315871b6978 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:45.773271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-45c51401-3cb2-4d56-969e-9315871b6978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:45.793319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f227c6-d8f3-4d68-b79e-5acf46b8e8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:45.794015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f227c6-d8f3-4d68-b79e-5acf46b8e8dc None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:45.794296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f227c6-d8f3-4d68-b79e-5acf46b8e8dc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:45.794573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f227c6-d8f3-4d68-b79e-5acf46b8e8dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:45.795623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f227c6-d8f3-4d68-b79e-5acf46b8e8dc None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=cfa25a63dd6f4ad08b0f858356528284) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:45.796145 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f227c6-d8f3-4d68-b79e-5acf46b8e8dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:45.797499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f227c6-d8f3-4d68-b79e-5acf46b8e8dc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:45.809681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 439/874] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:45 2022] GET /identity/v3/role_assignments?user.id=cfa25a63dd6f4ad08b0f858356528284&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:45.865765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 436/875] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:45.882791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f227c6-d8f3-4d68-b79e-5acf46b8e8dc None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:45.982623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:46.000594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f227c6-d8f3-4d68-b79e-5acf46b8e8dc None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:46.038649 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba55c0-c560-442c-8836-3d652006c44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:46.039361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba55c0-c560-442c-8836-3d652006c44c None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:46.039595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba55c0-c560-442c-8836-3d652006c44c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:46.039873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba55c0-c560-442c-8836-3d652006c44c None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:46.040596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba55c0-c560-442c-8836-3d652006c44c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:46.042000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba55c0-c560-442c-8836-3d652006c44c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:46.060374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 440/876] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:40:45 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:46.263317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ec185-6a00-47e5-9323-7e54d3505f19 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:46.263703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ec185-6a00-47e5-9323-7e54d3505f19 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:46.263884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ec185-6a00-47e5-9323-7e54d3505f19 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:46.266848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 437/877] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:46.272091 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2d8cf-d8e0-4e19-b9a9-d69f95fd953a None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:46.272379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2d8cf-d8e0-4e19-b9a9-d69f95fd953a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:46.272654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2d8cf-d8e0-4e19-b9a9-d69f95fd953a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:46.291201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-53c2d8cf-d8e0-4e19-b9a9-d69f95fd953a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:46.292374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-53c2d8cf-d8e0-4e19-b9a9-d69f95fd953a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:46.369289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 441/878] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:46.393034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ec185-6a00-47e5-9323-7e54d3505f19 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:46.425660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:46.501915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ec185-6a00-47e5-9323-7e54d3505f19 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:46.537727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f177dfd-a49f-47f3-b4e5-bf173729c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:46.538530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f177dfd-a49f-47f3-b4e5-bf173729c89a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:46.538862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f177dfd-a49f-47f3-b4e5-bf173729c89a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:46.539139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f177dfd-a49f-47f3-b4e5-bf173729c89a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:46.539794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f177dfd-a49f-47f3-b4e5-bf173729c89a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:46.541169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f177dfd-a49f-47f3-b4e5-bf173729c89a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-7f177dfd-a49f-47f3-b4e5-bf173729c89a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:46.554605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:46.555560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:46.556402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 438/879] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:40:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:46.562856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2d8cf-d8e0-4e19-b9a9-d69f95fd953a None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:46.636375 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:46.671909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2d8cf-d8e0-4e19-b9a9-d69f95fd953a None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:46.708395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ed33b-c40a-4512-80c7-a752db8a195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:46.709144 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ed33b-c40a-4512-80c7-a752db8a195e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:46.709344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ed33b-c40a-4512-80c7-a752db8a195e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:46.709604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ed33b-c40a-4512-80c7-a752db8a195e None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:46.710325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ed33b-c40a-4512-80c7-a752db8a195e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:46.710626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ed33b-c40a-4512-80c7-a752db8a195e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:46.711994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ed33b-c40a-4512-80c7-a752db8a195e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:46.717153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 442/880] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 1 19:40:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:46.723653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f177dfd-a49f-47f3-b4e5-bf173729c89a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:46.827690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f177dfd-a49f-47f3-b4e5-bf173729c89a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:46.835294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:46.864691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4190d7d-d445-4f05-a25a-a031c5534926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:46.865517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4190d7d-d445-4f05-a25a-a031c5534926 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:46.865833 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4190d7d-d445-4f05-a25a-a031c5534926 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:46.866147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4190d7d-d445-4f05-a25a-a031c5534926 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:46.866860 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4190d7d-d445-4f05-a25a-a031c5534926 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:46.867163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4190d7d-d445-4f05-a25a-a031c5534926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:46.868542 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4190d7d-d445-4f05-a25a-a031c5534926 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:46.874224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 439/881] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 1 19:40:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:47.106383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a4311-826a-46dd-94fe-db59062f8daf None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:47.106754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a4311-826a-46dd-94fe-db59062f8daf None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:47.107000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a4311-826a-46dd-94fe-db59062f8daf None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:47.108740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 443/882] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:47.114147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65282d-8b72-4e99-aa30-6a28e42f8fbc None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:47.114434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65282d-8b72-4e99-aa30-6a28e42f8fbc None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:47.114708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65282d-8b72-4e99-aa30-6a28e42f8fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:47.132207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7a65282d-8b72-4e99-aa30-6a28e42f8fbc None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:47.133387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7a65282d-8b72-4e99-aa30-6a28e42f8fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:47.204087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 440/883] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:47.227108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a4311-826a-46dd-94fe-db59062f8daf None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:47.347416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a4311-826a-46dd-94fe-db59062f8daf None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:47.383910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a9ece-949a-45f8-b441-09b24b98bd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:47.384707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9ece-949a-45f8-b441-09b24b98bd36 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:47.384987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9ece-949a-45f8-b441-09b24b98bd36 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:47.385263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9ece-949a-45f8-b441-09b24b98bd36 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:47.385945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a9ece-949a-45f8-b441-09b24b98bd36 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:47.387320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a9ece-949a-45f8-b441-09b24b98bd36 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:47.390493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-9a5a9ece-949a-45f8-b441-09b24b98bd36 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:47.400655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:47.401702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:47.402577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 444/884] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:40:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:47.408601 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65282d-8b72-4e99-aa30-6a28e42f8fbc None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:47.502266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:47.511830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65282d-8b72-4e99-aa30-6a28e42f8fbc None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:47.555006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751d9de-56f6-4bf6-abdc-c5ac68f0c6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:47.555672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d9de-56f6-4bf6-abdc-c5ac68f0c6cd None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:47.555952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d9de-56f6-4bf6-abdc-c5ac68f0c6cd None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:47.556225 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d9de-56f6-4bf6-abdc-c5ac68f0c6cd None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:47.556930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751d9de-56f6-4bf6-abdc-c5ac68f0c6cd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:47.557233 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751d9de-56f6-4bf6-abdc-c5ac68f0c6cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:47.558735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751d9de-56f6-4bf6-abdc-c5ac68f0c6cd None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:47.564260 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 441/885] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:40:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:47.569829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a9ece-949a-45f8-b441-09b24b98bd36 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:47.675559 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a9ece-949a-45f8-b441-09b24b98bd36 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:47.710942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4cc0a-bd63-4b49-875f-e2a2d93f96d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:47.711698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4cc0a-bd63-4b49-875f-e2a2d93f96d3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:47.711880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4cc0a-bd63-4b49-875f-e2a2d93f96d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:47.712157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4cc0a-bd63-4b49-875f-e2a2d93f96d3 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:47.712866 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4cc0a-bd63-4b49-875f-e2a2d93f96d3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:47.713173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4cc0a-bd63-4b49-875f-e2a2d93f96d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:47.714572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4cc0a-bd63-4b49-875f-e2a2d93f96d3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:47.720092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 445/886] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:40:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:47.725524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751d9de-56f6-4bf6-abdc-c5ac68f0c6cd None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:47.830612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751d9de-56f6-4bf6-abdc-c5ac68f0c6cd None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:47.867377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b91da-06b1-4edc-bb30-55234891dfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:47.868034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b91da-06b1-4edc-bb30-55234891dfec None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:47.868320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b91da-06b1-4edc-bb30-55234891dfec None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:47.868579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b91da-06b1-4edc-bb30-55234891dfec None admin] PATH_INFO: `/v3/endpoints` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:47.869362 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b91da-06b1-4edc-bb30-55234891dfec None admin] RBAC: Adding query filter params (interface=public, service_id=3c4d9cb33b524a5abc72bd68cdcc23ee, region_id=RegionOne) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:47.869700 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b91da-06b1-4edc-bb30-55234891dfec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:47.871081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b91da-06b1-4edc-bb30-55234891dfec None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:47.877409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 442/887] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 1 19:40:47 2022] GET /identity/v3/endpoints?service_id=3c4d9cb33b524a5abc72bd68cdcc23ee&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:47.882938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4cc0a-bd63-4b49-875f-e2a2d93f96d3 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:47.949636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:47.991379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4cc0a-bd63-4b49-875f-e2a2d93f96d3 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:48.024051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff14cb2-d77a-4ac8-804b-ee21040f2f63 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.024712 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff14cb2-d77a-4ac8-804b-ee21040f2f63 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.025226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff14cb2-d77a-4ac8-804b-ee21040f2f63 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.026921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 443/888] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:48.031355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe628-40bd-47c0-a51f-0d8a1bb74ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:48.032023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe628-40bd-47c0-a51f-0d8a1bb74ae9 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.032307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe628-40bd-47c0-a51f-0d8a1bb74ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.032582 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fe628-40bd-47c0-a51f-0d8a1bb74ae9 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.032899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668905-0c8c-485c-8b11-94591e20b8b4 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.033269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fe628-40bd-47c0-a51f-0d8a1bb74ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:48.033612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668905-0c8c-485c-8b11-94591e20b8b4 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.034106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668905-0c8c-485c-8b11-94591e20b8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.034824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fe628-40bd-47c0-a51f-0d8a1bb74ae9 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:48.041728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 446/889] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 1 19:40:47 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:40:48.053005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-34668905-0c8c-485c-8b11-94591e20b8b4 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:48.054671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-34668905-0c8c-485c-8b11-94591e20b8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:48.071129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:48.135202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 444/890] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:48.160829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe628-40bd-47c0-a51f-0d8a1bb74ae9 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:48.274771 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fe628-40bd-47c0-a51f-0d8a1bb74ae9 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:48.302154 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:48.310209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e27a95-2c85-41ce-b900-27342297183b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:48.311710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e27a95-2c85-41ce-b900-27342297183b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.312020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e27a95-2c85-41ce-b900-27342297183b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.312569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e27a95-2c85-41ce-b900-27342297183b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.323377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e27a95-2c85-41ce-b900-27342297183b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-65e27a95-2c85-41ce-b900-27342297183b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:48.331668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:48.332588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:48.333399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 447/891] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 1 19:40:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:48.338818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0671e2-267e-403d-84a5-47db81c77b56 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.339107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0671e2-267e-403d-84a5-47db81c77b56 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.339381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0671e2-267e-403d-84a5-47db81c77b56 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.341179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e27a95-2c85-41ce-b900-27342297183b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:48.342755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 445/892] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:48.347990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c477bd9-c326-493b-bf26-7c8e33d67f3c None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.348271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c477bd9-c326-493b-bf26-7c8e33d67f3c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.348544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c477bd9-c326-493b-bf26-7c8e33d67f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.365497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-2c477bd9-c326-493b-bf26-7c8e33d67f3c None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:48.367011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2c477bd9-c326-493b-bf26-7c8e33d67f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:48.455184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e27a95-2c85-41ce-b900-27342297183b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:48.461219 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 446/893] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:48.485132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c477bd9-c326-493b-bf26-7c8e33d67f3c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:48.491880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a99bb4-2ad6-43b5-9012-0721f5abdc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:48.492850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a99bb4-2ad6-43b5-9012-0721f5abdc06 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.493376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a99bb4-2ad6-43b5-9012-0721f5abdc06 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.493896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a99bb4-2ad6-43b5-9012-0721f5abdc06 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.494842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a99bb4-2ad6-43b5-9012-0721f5abdc06 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:48.495417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a99bb4-2ad6-43b5-9012-0721f5abdc06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:48.496991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a99bb4-2ad6-43b5-9012-0721f5abdc06 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:48.514796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 448/894] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 1 19:40:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:48.516318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a99bb4-2ad6-43b5-9012-0721f5abdc06 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:48.574418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:48.590932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:48.618994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c477bd9-c326-493b-bf26-7c8e33d67f3c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:48.634412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a99bb4-2ad6-43b5-9012-0721f5abdc06 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:48.670323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90408a9d-2a3a-422d-b1f0-c5036180cfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:48.671134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90408a9d-2a3a-422d-b1f0-c5036180cfa6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.671414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90408a9d-2a3a-422d-b1f0-c5036180cfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.671716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90408a9d-2a3a-422d-b1f0-c5036180cfa6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.672341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90408a9d-2a3a-422d-b1f0-c5036180cfa6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:48.673342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44661ad1-b6f5-4d52-a4f7-ec2aa323c7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:48.673749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90408a9d-2a3a-422d-b1f0-c5036180cfa6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:48.674617 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661ad1-b6f5-4d52-a4f7-ec2aa323c7c9 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.675106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661ad1-b6f5-4d52-a4f7-ec2aa323c7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.675629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661ad1-b6f5-4d52-a4f7-ec2aa323c7c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.687568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44661ad1-b6f5-4d52-a4f7-ec2aa323c7c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-90408a9d-2a3a-422d-b1f0-c5036180cfa6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:48.688622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:48.689513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:48.695380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 447/895] 10.222.0.66 () {56 vars in 1132 bytes} [Fri Jul 1 19:40:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:48.696210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44661ad1-b6f5-4d52-a4f7-ec2aa323c7c9 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:48.696282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90408a9d-2a3a-422d-b1f0-c5036180cfa6 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-44661ad1-b6f5-4d52-a4f7-ec2aa323c7c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:48.718984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:48.725205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:48.726193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:48.726193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:48.726193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:48.726193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:48.726193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 449/896] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:48.726463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44661ad1-b6f5-4d52-a4f7-ec2aa323c7c9 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:48.828594 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90408a9d-2a3a-422d-b1f0-c5036180cfa6 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:48.855298 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44661ad1-b6f5-4d52-a4f7-ec2aa323c7c9 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:48.856962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:48.878123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460cc15-5679-4bc9-b42c-4b7c2c80ca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:48.879116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460cc15-5679-4bc9-b42c-4b7c2c80ca34 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.880131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460cc15-5679-4bc9-b42c-4b7c2c80ca34 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.880629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460cc15-5679-4bc9-b42c-4b7c2c80ca34 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.881822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460cc15-5679-4bc9-b42c-4b7c2c80ca34 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:48.882574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460cc15-5679-4bc9-b42c-4b7c2c80ca34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:48.884846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460cc15-5679-4bc9-b42c-4b7c2c80ca34 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:48.892346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 448/897] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:40:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:48.897407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460cc15-5679-4bc9-b42c-4b7c2c80ca34 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:48.898727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9432b3-ecd8-46f0-8534-51b0099786b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:48.899401 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9432b3-ecd8-46f0-8534-51b0099786b1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:48.899679 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9432b3-ecd8-46f0-8534-51b0099786b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:48.899952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9432b3-ecd8-46f0-8534-51b0099786b1 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:48.900676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9432b3-ecd8-46f0-8534-51b0099786b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:48.900976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9432b3-ecd8-46f0-8534-51b0099786b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:48.903173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9432b3-ecd8-46f0-8534-51b0099786b1 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:48.909151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:48.919605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 450/898] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:48.922319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9432b3-ecd8-46f0-8534-51b0099786b1 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:49.014044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460cc15-5679-4bc9-b42c-4b7c2c80ca34 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:49.047520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9432b3-ecd8-46f0-8534-51b0099786b1 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:49.049065 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5abd1-8d52-4258-9bfc-b411b285df63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:49.049754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5abd1-8d52-4258-9bfc-b411b285df63 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:49.050040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5abd1-8d52-4258-9bfc-b411b285df63 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:49.050316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5abd1-8d52-4258-9bfc-b411b285df63 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:49.051020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5abd1-8d52-4258-9bfc-b411b285df63 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:49.051325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5abd1-8d52-4258-9bfc-b411b285df63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:49.052705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5abd1-8d52-4258-9bfc-b411b285df63 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:49.058211 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 449/899] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:40:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:49.059422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fe228-6221-4dbb-a578-a1a7f9abd0e5 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:49.059711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fe228-6221-4dbb-a578-a1a7f9abd0e5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:49.059987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fe228-6221-4dbb-a578-a1a7f9abd0e5 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:49.061987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 450/900] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:49.066661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fe228-6221-4dbb-a578-a1a7f9abd0e5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:49.087611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98ec26-f4f3-499c-9d5a-adf2e3b380aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:49.088342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98ec26-f4f3-499c-9d5a-adf2e3b380aa None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:49.088619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98ec26-f4f3-499c-9d5a-adf2e3b380aa None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:49.088893 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98ec26-f4f3-499c-9d5a-adf2e3b380aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:49.097559 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98ec26-f4f3-499c-9d5a-adf2e3b380aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:49.099017 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98ec26-f4f3-499c-9d5a-adf2e3b380aa None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-9b98ec26-f4f3-499c-9d5a-adf2e3b380aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:49.115494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:49.116553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:49.117638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:49.117638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:49.117638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:49.117638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:49.117638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:49.117638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:49.117638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:49.117638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:49.117638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:49.117638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:49.119909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 451/901] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 1 19:40:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:49.121325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e8f95-f8d0-44ad-a9a3-efb4fea84541 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:49.121798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e8f95-f8d0-44ad-a9a3-efb4fea84541 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:49.122221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e8f95-f8d0-44ad-a9a3-efb4fea84541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:49.139796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f10e8f95-f8d0-44ad-a9a3-efb4fea84541 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:49.140676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f10e8f95-f8d0-44ad-a9a3-efb4fea84541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:49.144714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:49.161200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:49.181834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fe228-6221-4dbb-a578-a1a7f9abd0e5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:49.216375 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 452/902] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:49.218288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e8f95-f8d0-44ad-a9a3-efb4fea84541 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:49.231371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04190be-6db4-4ef1-b765-1f98f6cd5e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:49.232531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04190be-6db4-4ef1-b765-1f98f6cd5e33 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:49.233561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04190be-6db4-4ef1-b765-1f98f6cd5e33 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:49.234610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04190be-6db4-4ef1-b765-1f98f6cd5e33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:49.236071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04190be-6db4-4ef1-b765-1f98f6cd5e33 None admin] RBAC: Adding query filter params (interface=public, service_id=3bd4c5a58c084e538b4a2ae8ea607efd, region_id=RegionOne) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:49.236533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04190be-6db4-4ef1-b765-1f98f6cd5e33 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:49.238580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04190be-6db4-4ef1-b765-1f98f6cd5e33 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:49.250154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 451/903] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 1 19:40:49 2022] GET /identity/v3/endpoints?service_id=3bd4c5a58c084e538b4a2ae8ea607efd&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:49.250154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04190be-6db4-4ef1-b765-1f98f6cd5e33 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:49.329450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e8f95-f8d0-44ad-a9a3-efb4fea84541 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:49.355901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04190be-6db4-4ef1-b765-1f98f6cd5e33 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:49.365055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b72dfa-70a3-4ba8-8f06-c788533c975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:49.365766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b72dfa-70a3-4ba8-8f06-c788533c975b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:49.366046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b72dfa-70a3-4ba8-8f06-c788533c975b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:49.366322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b72dfa-70a3-4ba8-8f06-c788533c975b None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:49.367164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b72dfa-70a3-4ba8-8f06-c788533c975b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:49.367473 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b72dfa-70a3-4ba8-8f06-c788533c975b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:49.368912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b72dfa-70a3-4ba8-8f06-c788533c975b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:49.388232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 453/904] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 1 19:40:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:49.389106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b72dfa-70a3-4ba8-8f06-c788533c975b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:49.401082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7129777-1be2-4cbb-9a0f-4bf8ab5062e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:49.401776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7129777-1be2-4cbb-9a0f-4bf8ab5062e4 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:49.402084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7129777-1be2-4cbb-9a0f-4bf8ab5062e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:49.402189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:49.402656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7129777-1be2-4cbb-9a0f-4bf8ab5062e4 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:49.403053 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7129777-1be2-4cbb-9a0f-4bf8ab5062e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:49.404967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7129777-1be2-4cbb-9a0f-4bf8ab5062e4 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:49.407198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:49.419631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 452/905] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:40:49 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:49.426701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7129777-1be2-4cbb-9a0f-4bf8ab5062e4 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:49.511311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b72dfa-70a3-4ba8-8f06-c788533c975b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:49.559588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90593017-105e-4092-9b25-18ec5e0ee558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:49.560644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593017-105e-4092-9b25-18ec5e0ee558 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:49.561195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593017-105e-4092-9b25-18ec5e0ee558 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:49.561851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593017-105e-4092-9b25-18ec5e0ee558 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:49.562506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7129777-1be2-4cbb-9a0f-4bf8ab5062e4 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:49.563051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90593017-105e-4092-9b25-18ec5e0ee558 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:49.564760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90593017-105e-4092-9b25-18ec5e0ee558 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:49.572775 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 454/906] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 1 19:40:49 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:40:49.608259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f6f66-b68d-4a77-89fd-9ba8b59ab40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:49.608259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f6f66-b68d-4a77-89fd-9ba8b59ab40e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:49.608862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f6f66-b68d-4a77-89fd-9ba8b59ab40e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:49.608862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f6f66-b68d-4a77-89fd-9ba8b59ab40e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:49.682410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:49.694636 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f6f66-b68d-4a77-89fd-9ba8b59ab40e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:49.696497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f6f66-b68d-4a77-89fd-9ba8b59ab40e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:49.805836 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 453/907] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:49 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:40:49.811935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90593017-105e-4092-9b25-18ec5e0ee558 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:49.885685 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:49.920097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90593017-105e-4092-9b25-18ec5e0ee558 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:49.955598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69006333-879c-497f-acc9-ed6c0e671047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:49.956399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69006333-879c-497f-acc9-ed6c0e671047 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:49.956673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69006333-879c-497f-acc9-ed6c0e671047 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:49.956946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69006333-879c-497f-acc9-ed6c0e671047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:49.968723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69006333-879c-497f-acc9-ed6c0e671047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:49.970378 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69006333-879c-497f-acc9-ed6c0e671047 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-69006333-879c-497f-acc9-ed6c0e671047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:49.988496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:49.989392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:49.990307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:49.990307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:49.990307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:49.990307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:49.990307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 455/908] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:49.996674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f6f66-b68d-4a77-89fd-9ba8b59ab40e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:50.093993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:50.103299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f6f66-b68d-4a77-89fd-9ba8b59ab40e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:50.144389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f333e6-8b60-4532-a70e-a39569eee683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:50.144971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f333e6-8b60-4532-a70e-a39569eee683 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:50.145249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f333e6-8b60-4532-a70e-a39569eee683 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:50.145524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f333e6-8b60-4532-a70e-a39569eee683 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:50.146278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f333e6-8b60-4532-a70e-a39569eee683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:50.146579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f333e6-8b60-4532-a70e-a39569eee683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:50.148065 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f333e6-8b60-4532-a70e-a39569eee683 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:50.160935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 454/909] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:50.166692 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69006333-879c-497f-acc9-ed6c0e671047 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:50.196117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf3363-74c9-483d-b072-06b5941f9951 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:50.196773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf3363-74c9-483d-b072-06b5941f9951 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:50.197263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf3363-74c9-483d-b072-06b5941f9951 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:50.199182 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 455/910] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:50.206488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b0366-6241-40e0-879f-78587ec659fd None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:50.207032 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b0366-6241-40e0-879f-78587ec659fd None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:50.207543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b0366-6241-40e0-879f-78587ec659fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:50.225274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-931b0366-6241-40e0-879f-78587ec659fd None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:50.227043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-931b0366-6241-40e0-879f-78587ec659fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:50.287595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69006333-879c-497f-acc9-ed6c0e671047 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:50.309630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 456/911] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:50.317159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:50.328029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6a2b0-9089-491e-97c6-f4f120c99b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:50.328832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6a2b0-9089-491e-97c6-f4f120c99b0b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:50.329111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6a2b0-9089-491e-97c6-f4f120c99b0b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:50.329388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6a2b0-9089-491e-97c6-f4f120c99b0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:50.333503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b0366-6241-40e0-879f-78587ec659fd None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:50.343088 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6a2b0-9089-491e-97c6-f4f120c99b0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:50.344921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6a2b0-9089-491e-97c6-f4f120c99b0b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-55d6a2b0-9089-491e-97c6-f4f120c99b0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:50.361179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:50.362316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:50.363304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:50.363304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:50.363304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:50.363304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:50.363304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:50.363304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:50.363304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:50.363304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:50.371017 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 456/912] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:50.372240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6a2b0-9089-491e-97c6-f4f120c99b0b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:50.425181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:50.446752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b0366-6241-40e0-879f-78587ec659fd None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:50.489314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306410b8-33a1-49c2-9599-fd7d3236fc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:50.490141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306410b8-33a1-49c2-9599-fd7d3236fc41 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:50.490426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306410b8-33a1-49c2-9599-fd7d3236fc41 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:50.490702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306410b8-33a1-49c2-9599-fd7d3236fc41 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:50.491354 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306410b8-33a1-49c2-9599-fd7d3236fc41 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:50.492734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306410b8-33a1-49c2-9599-fd7d3236fc41 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-306410b8-33a1-49c2-9599-fd7d3236fc41 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:50.507611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:50.508693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:50.509543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 457/913] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:40:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:50.516640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306410b8-33a1-49c2-9599-fd7d3236fc41 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:50.521730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6a2b0-9089-491e-97c6-f4f120c99b0b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:50.564004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b793497-947b-4502-b9b3-5b827309d0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:50.564762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b793497-947b-4502-b9b3-5b827309d0de None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:50.565047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b793497-947b-4502-b9b3-5b827309d0de None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:50.565325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b793497-947b-4502-b9b3-5b827309d0de None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:50.566149 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b793497-947b-4502-b9b3-5b827309d0de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:50.566457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b793497-947b-4502-b9b3-5b827309d0de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:50.567894 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b793497-947b-4502-b9b3-5b827309d0de None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:50.581613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 457/914] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:50.587494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b793497-947b-4502-b9b3-5b827309d0de None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:50.637105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:50.659196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306410b8-33a1-49c2-9599-fd7d3236fc41 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:50.703121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b793497-947b-4502-b9b3-5b827309d0de None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:50.709039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210f332-d853-4717-ac07-49379eb762be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:50.709928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210f332-d853-4717-ac07-49379eb762be None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:50.710374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210f332-d853-4717-ac07-49379eb762be None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:50.710808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210f332-d853-4717-ac07-49379eb762be None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:50.712205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210f332-d853-4717-ac07-49379eb762be None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:50.712666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210f332-d853-4717-ac07-49379eb762be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:50.714675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210f332-d853-4717-ac07-49379eb762be None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:50.726311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 458/915] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:40:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:50.728069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210f332-d853-4717-ac07-49379eb762be None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:50.743244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:50.746482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f73d7f-b35f-4bff-b20c-56868c081aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:50.747166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73d7f-b35f-4bff-b20c-56868c081aa1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:50.747547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73d7f-b35f-4bff-b20c-56868c081aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:50.747924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73d7f-b35f-4bff-b20c-56868c081aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:50.749903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f73d7f-b35f-4bff-b20c-56868c081aa1 None admin] RBAC: Adding query filter params (role.id=0f21ddda19344dfd855c441dd59142ef, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=cfa25a63dd6f4ad08b0f858356528284) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:50.750762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f73d7f-b35f-4bff-b20c-56868c081aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:50.752648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f73d7f-b35f-4bff-b20c-56868c081aa1 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:50.765468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 458/916] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:50 2022] GET /identity/v3/role_assignments?user.id=cfa25a63dd6f4ad08b0f858356528284&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0f21ddda19344dfd855c441dd59142ef => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:50.855458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210f332-d853-4717-ac07-49379eb762be None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:50.894530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b65a2-58c9-40b2-8f4f-f3895be760ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:50.895197 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b65a2-58c9-40b2-8f4f-f3895be760ed None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:50.895472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b65a2-58c9-40b2-8f4f-f3895be760ed None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:50.895739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b65a2-58c9-40b2-8f4f-f3895be760ed None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:50.896447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b65a2-58c9-40b2-8f4f-f3895be760ed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:50.896745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b65a2-58c9-40b2-8f4f-f3895be760ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:50.898155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b65a2-58c9-40b2-8f4f-f3895be760ed None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:50.904204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 459/917] 10.222.0.66 () {56 vars in 1114 bytes} [Fri Jul 1 19:40:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:50.910895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f73d7f-b35f-4bff-b20c-56868c081aa1 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:51.020878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f73d7f-b35f-4bff-b20c-56868c081aa1 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:51.028973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:51.058451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f22be-1e9b-4c66-983a-de41e38f99a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:51.059470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f22be-1e9b-4c66-983a-de41e38f99a5 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:51.059966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f22be-1e9b-4c66-983a-de41e38f99a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:51.060469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f22be-1e9b-4c66-983a-de41e38f99a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:51.061464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f22be-1e9b-4c66-983a-de41e38f99a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:51.063216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f22be-1e9b-4c66-983a-de41e38f99a5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:51.088127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 459/918] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 1 19:40:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:51.606748 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab74ea-740d-4da9-81c0-c8dde504e9af None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:51.607088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab74ea-740d-4da9-81c0-c8dde504e9af None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:51.607302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab74ea-740d-4da9-81c0-c8dde504e9af None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:51.609573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 460/919] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:51.614922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0db68-f1c1-4140-9979-09d82b661a26 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:51.615210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0db68-f1c1-4140-9979-09d82b661a26 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:51.615488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0db68-f1c1-4140-9979-09d82b661a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:51.633055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-61c0db68-f1c1-4140-9979-09d82b661a26 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:51.634276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-61c0db68-f1c1-4140-9979-09d82b661a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:51.710647 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 460/920] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:51.734376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab74ea-740d-4da9-81c0-c8dde504e9af None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:51.817016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:51.832952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83429061-78cb-4197-9095-282dfb457d77 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:51.833281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83429061-78cb-4197-9095-282dfb457d77 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:51.833500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83429061-78cb-4197-9095-282dfb457d77 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:51.835010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 461/921] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:51.840000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2a23a-eee9-4e96-a4a8-d10fba4c0071 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:51.840541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2a23a-eee9-4e96-a4a8-d10fba4c0071 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:51.841040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2a23a-eee9-4e96-a4a8-d10fba4c0071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:51.849341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab74ea-740d-4da9-81c0-c8dde504e9af None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:51.859244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7cb2a23a-eee9-4e96-a4a8-d10fba4c0071 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:51.860707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7cb2a23a-eee9-4e96-a4a8-d10fba4c0071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:51.890867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7ec9b-6320-4d3f-8391-d6672fe19b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:51.891977 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7ec9b-6320-4d3f-8391-d6672fe19b72 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:51.892260 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7ec9b-6320-4d3f-8391-d6672fe19b72 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:51.892770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7ec9b-6320-4d3f-8391-d6672fe19b72 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:51.893427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7ec9b-6320-4d3f-8391-d6672fe19b72 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:51.894996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7ec9b-6320-4d3f-8391-d6672fe19b72 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-c0a7ec9b-6320-4d3f-8391-d6672fe19b72 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:51.911230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:51.912427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:51.913414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:51.916309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 461/922] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:40:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:51.922218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7ec9b-6320-4d3f-8391-d6672fe19b72 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:51.941704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 462/923] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:51.963717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2a23a-eee9-4e96-a4a8-d10fba4c0071 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:51.989937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:52.057714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7ec9b-6320-4d3f-8391-d6672fe19b72 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:52.080111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2a23a-eee9-4e96-a4a8-d10fba4c0071 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:52.103227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627ffd-b7c3-46d3-ae00-e4c7a240be7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:52.104377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627ffd-b7c3-46d3-ae00-e4c7a240be7a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:52.104846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627ffd-b7c3-46d3-ae00-e4c7a240be7a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:52.105573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db627ffd-b7c3-46d3-ae00-e4c7a240be7a None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:52.106784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db627ffd-b7c3-46d3-ae00-e4c7a240be7a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:52.107394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db627ffd-b7c3-46d3-ae00-e4c7a240be7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:52.110340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db627ffd-b7c3-46d3-ae00-e4c7a240be7a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:52.113139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:52.116985 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94348725-807d-45ed-8b12-ebe43c09023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:52.117672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 462/924] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 1 19:40:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:52.122323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94348725-807d-45ed-8b12-ebe43c09023d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:52.122504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627ffd-b7c3-46d3-ae00-e4c7a240be7a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:52.122958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94348725-807d-45ed-8b12-ebe43c09023d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:52.123602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94348725-807d-45ed-8b12-ebe43c09023d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:52.124818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94348725-807d-45ed-8b12-ebe43c09023d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:52.128218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94348725-807d-45ed-8b12-ebe43c09023d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-94348725-807d-45ed-8b12-ebe43c09023d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:52.145018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:52.145999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 19:40:52.146856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:52.150249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 463/925] 10.222.0.66 () {56 vars in 1132 bytes} [Fri Jul 1 19:40:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:52.156757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94348725-807d-45ed-8b12-ebe43c09023d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:52.252877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db627ffd-b7c3-46d3-ae00-e4c7a240be7a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:52.262869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:52.288763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94348725-807d-45ed-8b12-ebe43c09023d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:52.301510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890387-f462-4f47-a450-b1528abe481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:52.302814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890387-f462-4f47-a450-b1528abe481e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:52.303534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890387-f462-4f47-a450-b1528abe481e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:52.304010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890387-f462-4f47-a450-b1528abe481e None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:52.305106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd890387-f462-4f47-a450-b1528abe481e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:52.305556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd890387-f462-4f47-a450-b1528abe481e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:52.307768 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd890387-f462-4f47-a450-b1528abe481e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:52.315857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 463/926] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 1 19:40:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:52.321262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890387-f462-4f47-a450-b1528abe481e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:52.334458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5af40d-f178-4029-af59-3285dd3177b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:52.335836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af40d-f178-4029-af59-3285dd3177b6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:52.336331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af40d-f178-4029-af59-3285dd3177b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:52.337178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af40d-f178-4029-af59-3285dd3177b6 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:52.338487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5af40d-f178-4029-af59-3285dd3177b6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:52.338945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5af40d-f178-4029-af59-3285dd3177b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:52.341281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5af40d-f178-4029-af59-3285dd3177b6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:52.348403 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 464/927] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:40:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:52.354167 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5af40d-f178-4029-af59-3285dd3177b6 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:52.393907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:52.441766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890387-f462-4f47-a450-b1528abe481e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:52.480063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5af40d-f178-4029-af59-3285dd3177b6 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:52.489969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959884e-3ca9-4baf-a24d-039111f1d057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:52.491301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959884e-3ca9-4baf-a24d-039111f1d057 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:52.491754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959884e-3ca9-4baf-a24d-039111f1d057 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:52.492586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959884e-3ca9-4baf-a24d-039111f1d057 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:52.493844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959884e-3ca9-4baf-a24d-039111f1d057 None admin] RBAC: Adding query filter params (interface=public, service_id=175a123f146647639c4c9f3fff0603c8, region_id=RegionOne) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:52.494326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959884e-3ca9-4baf-a24d-039111f1d057 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:52.496570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959884e-3ca9-4baf-a24d-039111f1d057 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:52.509360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 464/928] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 1 19:40:52 2022] GET /identity/v3/endpoints?service_id=175a123f146647639c4c9f3fff0603c8&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:52.515068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959884e-3ca9-4baf-a24d-039111f1d057 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:52.527435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a7ba4-f210-426d-9ce2-fb3aa9034245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:52.529083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a7ba4-f210-426d-9ce2-fb3aa9034245 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:52.530496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a7ba4-f210-426d-9ce2-fb3aa9034245 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:52.530988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a7ba4-f210-426d-9ce2-fb3aa9034245 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:52.532153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a7ba4-f210-426d-9ce2-fb3aa9034245 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:52.532654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a7ba4-f210-426d-9ce2-fb3aa9034245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:52.535323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a7ba4-f210-426d-9ce2-fb3aa9034245 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:52.543077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 465/929] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:40:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:52.550392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a7ba4-f210-426d-9ce2-fb3aa9034245 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:52.570315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:52.657475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959884e-3ca9-4baf-a24d-039111f1d057 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:52.686412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:52.688397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a7ba4-f210-426d-9ce2-fb3aa9034245 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:52.694890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b1e2b-9af1-4c3f-b468-96086722828b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:52.695559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b1e2b-9af1-4c3f-b468-96086722828b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:52.695840 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b1e2b-9af1-4c3f-b468-96086722828b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:52.696111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b1e2b-9af1-4c3f-b468-96086722828b None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:52.696795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b1e2b-9af1-4c3f-b468-96086722828b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:52.698688 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b1e2b-9af1-4c3f-b468-96086722828b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:52.710445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 465/930] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 1 19:40:52 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:40:52.734223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb57f48-0d33-4344-b79a-725cb4c5a402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:52.734950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb57f48-0d33-4344-b79a-725cb4c5a402 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:52.735234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb57f48-0d33-4344-b79a-725cb4c5a402 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:52.735510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb57f48-0d33-4344-b79a-725cb4c5a402 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:52.736278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb57f48-0d33-4344-b79a-725cb4c5a402 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:52.737795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb57f48-0d33-4344-b79a-725cb4c5a402 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:52.761096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 466/931] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 1 19:40:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:52.938847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71939d9-e98f-41d8-b04f-0454e3bb37b8 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:52.939365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71939d9-e98f-41d8-b04f-0454e3bb37b8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:52.939365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71939d9-e98f-41d8-b04f-0454e3bb37b8 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:52.940911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 466/932] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:52.946199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f4122-f0c7-4b24-8693-3b78e1b30196 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:52.946615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f4122-f0c7-4b24-8693-3b78e1b30196 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:52.946772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f4122-f0c7-4b24-8693-3b78e1b30196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:52.965526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f98f4122-f0c7-4b24-8693-3b78e1b30196 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:52.966805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f98f4122-f0c7-4b24-8693-3b78e1b30196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:53.033363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:53.048681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 467/933] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:53.071645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71939d9-e98f-41d8-b04f-0454e3bb37b8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:53.182983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71939d9-e98f-41d8-b04f-0454e3bb37b8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:53.220540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81969383-73e5-49f1-8f11-f8df3e326b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:53.221277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81969383-73e5-49f1-8f11-f8df3e326b73 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:53.221556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81969383-73e5-49f1-8f11-f8df3e326b73 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:53.221854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81969383-73e5-49f1-8f11-f8df3e326b73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:53.232590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81969383-73e5-49f1-8f11-f8df3e326b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-81969383-73e5-49f1-8f11-f8df3e326b73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:53.241541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:53.242536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:53.243365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 467/934] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 1 19:40:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:53.251403 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f4122-f0c7-4b24-8693-3b78e1b30196 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:53.260739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e1153-3baa-49e1-986a-b86a6732080b None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:53.260739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e1153-3baa-49e1-986a-b86a6732080b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:53.261096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e1153-3baa-49e1-986a-b86a6732080b None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:53.262537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 468/935] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:53.267871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e2fa2-a3db-4ef4-a315-28202536920f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:53.268150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e2fa2-a3db-4ef4-a315-28202536920f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:53.268425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e2fa2-a3db-4ef4-a315-28202536920f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:53.288900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-4a3e2fa2-a3db-4ef4-a315-28202536920f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:53.290134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4a3e2fa2-a3db-4ef4-a315-28202536920f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:53.341685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:53.369892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f4122-f0c7-4b24-8693-3b78e1b30196 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:53.370274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 469/936] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:53.391053 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e2fa2-a3db-4ef4-a315-28202536920f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:53.405332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39ec24-c585-4d0d-88eb-7df9850db96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:53.406031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39ec24-c585-4d0d-88eb-7df9850db96d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:53.406339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39ec24-c585-4d0d-88eb-7df9850db96d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:53.406578 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39ec24-c585-4d0d-88eb-7df9850db96d None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:53.407325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39ec24-c585-4d0d-88eb-7df9850db96d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:53.407629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39ec24-c585-4d0d-88eb-7df9850db96d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:53.409001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39ec24-c585-4d0d-88eb-7df9850db96d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:53.422385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 468/937] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 1 19:40:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:53.423869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39ec24-c585-4d0d-88eb-7df9850db96d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:53.496273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:53.525444 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e2fa2-a3db-4ef4-a315-28202536920f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:53.564541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:53.565232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:53.565511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:53.565824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:53.566620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:53.568058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:53.569411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: WARNING keystone.server.flask.common [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 01 19:40:53.572105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39ec24-c585-4d0d-88eb-7df9850db96d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:53.587965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.sql.core [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 19:40:53.587965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 01 19:40:53.587965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [parameters: {'id': '6dfb11057dd444919bf72dc40d549c8d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 01 19:40:53.587965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86779) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6dfb11057dd444919bf72dc40d549c8d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6dfb11057dd444919bf72dc40d549c8d. Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application conn.query(q) Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 19:40:53.597187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result.read() Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 01 19:40:53.598055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application next(self.gen) Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application yield resource Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application next(self.gen) Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise self.value Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application session.commit() Jul 01 19:40:53.598993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.session.flush() Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application compat.raise_( Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 19:40:53.599882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 19:40:53.600801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application conn.query(q) Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result.read() Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 19:40:53.601619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [parameters: {'id': '6dfb11057dd444919bf72dc40d549c8d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:53.602494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:53.603301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:53.604213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:53.604213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 01 19:40:53.604213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 01 19:40:53.604213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:53.604213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:53.604213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 01 19:40:53.604213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 01 19:40:53.604213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6dfb11057dd444919bf72dc40d549c8d. Jul 01 19:40:53.604213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:53.604213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 470/938] 10.222.0.66 () {60 vars in 1140 bytes} [Fri Jul 1 19:40:53 2022] POST /identity/v3/projects => generated 262 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:53.604781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:53.627570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073d77c-7c14-4b20-8b24-b148095cf728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:53.628733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073d77c-7c14-4b20-8b24-b148095cf728 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:53.629243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073d77c-7c14-4b20-8b24-b148095cf728 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:53.629776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073d77c-7c14-4b20-8b24-b148095cf728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:53.645784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073d77c-7c14-4b20-8b24-b148095cf728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:53.647532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073d77c-7c14-4b20-8b24-b148095cf728 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:53.650418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-7073d77c-7c14-4b20-8b24-b148095cf728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:53.671483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:53.672424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:53.673292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:53.673292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:53.673292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:53.673292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:53.679971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 469/939] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:53.680573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073d77c-7c14-4b20-8b24-b148095cf728 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:53.727269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b489ca-2e1f-4ae3-8c69-390feb36961a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:53.765875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846deb1-338f-4fd6-a739-ee75adf8a3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:53.766750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846deb1-338f-4fd6-a739-ee75adf8a3d6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:53.767028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846deb1-338f-4fd6-a739-ee75adf8a3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:53.767298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846deb1-338f-4fd6-a739-ee75adf8a3d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:53.774769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846deb1-338f-4fd6-a739-ee75adf8a3d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:53.776468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846deb1-338f-4fd6-a739-ee75adf8a3d6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-5846deb1-338f-4fd6-a739-ee75adf8a3d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:53.791420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:53.792311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:53.793187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:53.793187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:53.793187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:53.793187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:53.793187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:53.793187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:53.793187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:53.793187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:53.797707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 471/940] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:53.798707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846deb1-338f-4fd6-a739-ee75adf8a3d6 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:53.808898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073d77c-7c14-4b20-8b24-b148095cf728 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:53.818854 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:53.853486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9094b9c-e738-4b38-ad3d-891b7d8af37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:53.854462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9094b9c-e738-4b38-ad3d-891b7d8af37f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:53.854890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9094b9c-e738-4b38-ad3d-891b7d8af37f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:53.855179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9094b9c-e738-4b38-ad3d-891b7d8af37f None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:53.855909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9094b9c-e738-4b38-ad3d-891b7d8af37f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:53.856581 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9094b9c-e738-4b38-ad3d-891b7d8af37f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:53.858591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9094b9c-e738-4b38-ad3d-891b7d8af37f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:53.877653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 470/941] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:53.883852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9094b9c-e738-4b38-ad3d-891b7d8af37f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:53.924363 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:53.929167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846deb1-338f-4fd6-a739-ee75adf8a3d6 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:53.971615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc013-6fb0-493a-915a-df7023a7cb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:53.973120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc013-6fb0-493a-915a-df7023a7cb38 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:53.973706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc013-6fb0-493a-915a-df7023a7cb38 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:53.974198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfc013-6fb0-493a-915a-df7023a7cb38 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:53.975265 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfc013-6fb0-493a-915a-df7023a7cb38 None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:53.975803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfc013-6fb0-493a-915a-df7023a7cb38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:53.977523 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abfc013-6fb0-493a-915a-df7023a7cb38 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:54.005648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9094b9c-e738-4b38-ad3d-891b7d8af37f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:54.006706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 472/942] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:40:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:54.057361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d854d2-9774-4321-a90b-d0213301733e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:54.058606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d854d2-9774-4321-a90b-d0213301733e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:54.059057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d854d2-9774-4321-a90b-d0213301733e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:54.059497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d854d2-9774-4321-a90b-d0213301733e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:54.072298 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d854d2-9774-4321-a90b-d0213301733e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:54.073800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d854d2-9774-4321-a90b-d0213301733e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-26d854d2-9774-4321-a90b-d0213301733e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:54.090519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:54.091530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:54.093020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 471/943] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 1 19:40:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:54.098060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc013-6fb0-493a-915a-df7023a7cb38 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:54.219231 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfc013-6fb0-493a-915a-df7023a7cb38 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:54.255186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e0a6b-a977-478b-94db-4156265ae600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:54.255858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e0a6b-a977-478b-94db-4156265ae600 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:54.256138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e0a6b-a977-478b-94db-4156265ae600 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:54.256421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e0a6b-a977-478b-94db-4156265ae600 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:54.257258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e0a6b-a977-478b-94db-4156265ae600 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:54.257567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e0a6b-a977-478b-94db-4156265ae600 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:54.259027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e0a6b-a977-478b-94db-4156265ae600 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:54.278517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 473/944] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 1 19:40:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:54.284285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d854d2-9774-4321-a90b-d0213301733e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:54.313034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:54.398872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d854d2-9774-4321-a90b-d0213301733e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:54.434528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f064f-6a0b-49eb-bf06-92b4ee16b196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:54.435331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f064f-6a0b-49eb-bf06-92b4ee16b196 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:54.436147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f064f-6a0b-49eb-bf06-92b4ee16b196 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:54.436466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f064f-6a0b-49eb-bf06-92b4ee16b196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:54.447720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f064f-6a0b-49eb-bf06-92b4ee16b196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:54.449193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f064f-6a0b-49eb-bf06-92b4ee16b196 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-db5f064f-6a0b-49eb-bf06-92b4ee16b196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:54.468024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:54.468966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:54.469860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:54.469860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:54.469860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:54.469860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:54.469860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 472/945] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:54.475857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e0a6b-a977-478b-94db-4156265ae600 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:54.518000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:54.596085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e0a6b-a977-478b-94db-4156265ae600 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:54.631794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cc2b8-e92b-431d-b37c-be59f977d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:54.632829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cc2b8-e92b-431d-b37c-be59f977d241 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:54.633315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cc2b8-e92b-431d-b37c-be59f977d241 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:54.633844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cc2b8-e92b-431d-b37c-be59f977d241 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:54.634765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333cc2b8-e92b-431d-b37c-be59f977d241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:54.635302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333cc2b8-e92b-431d-b37c-be59f977d241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:54.636890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333cc2b8-e92b-431d-b37c-be59f977d241 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:54.653982 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 474/946] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:54.655828 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f064f-6a0b-49eb-bf06-92b4ee16b196 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:54.746310 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:54.770331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f064f-6a0b-49eb-bf06-92b4ee16b196 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:54.805676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822e8ab-57db-49e7-ad8e-cba09fe220fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:54.806487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822e8ab-57db-49e7-ad8e-cba09fe220fb None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:54.806768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822e8ab-57db-49e7-ad8e-cba09fe220fb None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:54.807044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822e8ab-57db-49e7-ad8e-cba09fe220fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:54.814912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822e8ab-57db-49e7-ad8e-cba09fe220fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:54.816378 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822e8ab-57db-49e7-ad8e-cba09fe220fb None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6822e8ab-57db-49e7-ad8e-cba09fe220fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:54.831426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:54.832343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:54.833225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:54.833225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:54.833225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:54.833225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:54.833225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:54.833225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:54.833225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:54.833225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:54.833225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 473/947] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:54.840011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cc2b8-e92b-431d-b37c-be59f977d241 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:54.922770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:54.947025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cc2b8-e92b-431d-b37c-be59f977d241 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:54.955350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc36e24-b977-4d6a-b312-4e462c9748ff None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:54.955531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc36e24-b977-4d6a-b312-4e462c9748ff None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:54.955731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc36e24-b977-4d6a-b312-4e462c9748ff None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:54.957321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 474/948] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:54.962174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f973a4d-15c8-4128-aaef-0325dda0cb59 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:54.962460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f973a4d-15c8-4128-aaef-0325dda0cb59 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:54.962740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f973a4d-15c8-4128-aaef-0325dda0cb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:54.980651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9f973a4d-15c8-4128-aaef-0325dda0cb59 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:54.981944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9f973a4d-15c8-4128-aaef-0325dda0cb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:54.985556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d5c9e-cee1-4e7a-961a-5641cfe2767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:54.986759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d5c9e-cee1-4e7a-961a-5641cfe2767c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:54.987037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d5c9e-cee1-4e7a-961a-5641cfe2767c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:54.987306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d5c9e-cee1-4e7a-961a-5641cfe2767c None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:54.988095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d5c9e-cee1-4e7a-961a-5641cfe2767c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:54.988401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d5c9e-cee1-4e7a-961a-5641cfe2767c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:54.989863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d5c9e-cee1-4e7a-961a-5641cfe2767c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:55.004141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 475/949] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:55.005322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c4d02-6934-4a5c-8405-29c5a21211ba None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:55.009683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c4d02-6934-4a5c-8405-29c5a21211ba None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:55.010319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c4d02-6934-4a5c-8405-29c5a21211ba None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:55.013916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 476/950] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:55.014598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c4d02-6934-4a5c-8405-29c5a21211ba None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:55.065667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 475/951] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:55.066193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42491d1-5d5e-42a0-b2f2-cff6a8c75721 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:55.066193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42491d1-5d5e-42a0-b2f2-cff6a8c75721 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:55.066356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42491d1-5d5e-42a0-b2f2-cff6a8c75721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:55.092707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c42491d1-5d5e-42a0-b2f2-cff6a8c75721 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:55.093886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c42491d1-5d5e-42a0-b2f2-cff6a8c75721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:55.135994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c4d02-6934-4a5c-8405-29c5a21211ba None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:55.161173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:55.168056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 476/952] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:55.168750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42491d1-5d5e-42a0-b2f2-cff6a8c75721 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:55.184311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940327da-bbf4-4356-bb4d-3245d0d2f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:55.186803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940327da-bbf4-4356-bb4d-3245d0d2f74c None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:55.187293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940327da-bbf4-4356-bb4d-3245d0d2f74c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:55.188109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940327da-bbf4-4356-bb4d-3245d0d2f74c None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/cfa25a63dd6f4ad08b0f858356528284/roles/0f21ddda19344dfd855c441dd59142ef` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:55.191485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940327da-bbf4-4356-bb4d-3245d0d2f74c None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=cfa25a63dd6f4ad08b0f858356528284, role_id=0f21ddda19344dfd855c441dd59142ef)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:55.193967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940327da-bbf4-4356-bb4d-3245d0d2f74c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:55.198621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:55.213868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 477/953] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:40:55 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/cfa25a63dd6f4ad08b0f858356528284/roles/0f21ddda19344dfd855c441dd59142ef => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:40:55.215455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940327da-bbf4-4356-bb4d-3245d0d2f74c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:55.282718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42491d1-5d5e-42a0-b2f2-cff6a8c75721 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:55.329487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93269036-3a73-4514-b9c0-87c4ae470301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:55.330369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269036-3a73-4514-b9c0-87c4ae470301 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:55.330649 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269036-3a73-4514-b9c0-87c4ae470301 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:55.330918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269036-3a73-4514-b9c0-87c4ae470301 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:55.334096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93269036-3a73-4514-b9c0-87c4ae470301 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:55.335759 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93269036-3a73-4514-b9c0-87c4ae470301 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-93269036-3a73-4514-b9c0-87c4ae470301 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:55.354073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:55.355125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:55.356127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:55.359845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 477/954] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:40:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:55.365527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93269036-3a73-4514-b9c0-87c4ae470301 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:55.380038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940327da-bbf4-4356-bb4d-3245d0d2f74c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:55.429176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff57670-917f-40e1-bc1c-ead1b090678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:55.430062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff57670-917f-40e1-bc1c-ead1b090678d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:55.430336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff57670-917f-40e1-bc1c-ead1b090678d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:55.431023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff57670-917f-40e1-bc1c-ead1b090678d None admin] PATH_INFO: `/v3/services/compute` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:55.431685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff57670-917f-40e1-bc1c-ead1b090678d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:55.433404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff57670-917f-40e1-bc1c-ead1b090678d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-5ff57670-917f-40e1-bc1c-ead1b090678d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:55.451220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:55.453179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:55.454484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 478/955] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:55.457918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:55.460891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff57670-917f-40e1-bc1c-ead1b090678d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:55.493455 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93269036-3a73-4514-b9c0-87c4ae470301 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:55.534015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:55.539244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b5033-0891-4254-a9e9-eeea27b7bb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:55.539910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b5033-0891-4254-a9e9-eeea27b7bb0f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:55.540234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b5033-0891-4254-a9e9-eeea27b7bb0f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:55.540513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b5033-0891-4254-a9e9-eeea27b7bb0f None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:55.541277 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b5033-0891-4254-a9e9-eeea27b7bb0f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:55.541593 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b5033-0891-4254-a9e9-eeea27b7bb0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:55.543036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b5033-0891-4254-a9e9-eeea27b7bb0f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:55.549194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 478/956] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 1 19:40:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:55.554726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b5033-0891-4254-a9e9-eeea27b7bb0f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:55.583063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff57670-917f-40e1-bc1c-ead1b090678d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:55.628100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc5be6-494f-459c-b39b-11b968cbd9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:55.628772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5be6-494f-459c-b39b-11b968cbd9e5 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:55.629051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5be6-494f-459c-b39b-11b968cbd9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:55.629321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5be6-494f-459c-b39b-11b968cbd9e5 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:55.630064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc5be6-494f-459c-b39b-11b968cbd9e5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:55.630420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc5be6-494f-459c-b39b-11b968cbd9e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:55.632060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc5be6-494f-459c-b39b-11b968cbd9e5 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:55.645356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 479/957] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:40:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:55.646453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc5be6-494f-459c-b39b-11b968cbd9e5 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:55.672986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b5033-0891-4254-a9e9-eeea27b7bb0f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:55.704349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:55.715306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658db73f-8626-4015-86c7-651766277e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:55.716307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658db73f-8626-4015-86c7-651766277e27 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:55.716856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658db73f-8626-4015-86c7-651766277e27 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:55.718151 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658db73f-8626-4015-86c7-651766277e27 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:55.719236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658db73f-8626-4015-86c7-651766277e27 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:55.719739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658db73f-8626-4015-86c7-651766277e27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:55.721435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658db73f-8626-4015-86c7-651766277e27 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:55.727463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 479/958] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 1 19:40:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:55.735789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658db73f-8626-4015-86c7-651766277e27 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:55.769502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc5be6-494f-459c-b39b-11b968cbd9e5 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:55.800807 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:55.816361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c92a9-7fe0-42ce-a64b-c9e19903550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:55.817461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c92a9-7fe0-42ce-a64b-c9e19903550c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:55.817904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c92a9-7fe0-42ce-a64b-c9e19903550c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:55.818345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c92a9-7fe0-42ce-a64b-c9e19903550c None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:55.819249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c92a9-7fe0-42ce-a64b-c9e19903550c None admin] RBAC: Adding query filter params (type=compute) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:55.819755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c92a9-7fe0-42ce-a64b-c9e19903550c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:55.821474 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c92a9-7fe0-42ce-a64b-c9e19903550c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:55.834122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 480/959] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:40:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:55.860789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658db73f-8626-4015-86c7-651766277e27 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:55.899491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55a12b-2ae6-4576-8d52-dd9495425329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:55.900158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55a12b-2ae6-4576-8d52-dd9495425329 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:55.900443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55a12b-2ae6-4576-8d52-dd9495425329 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:55.900722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55a12b-2ae6-4576-8d52-dd9495425329 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:55.901488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55a12b-2ae6-4576-8d52-dd9495425329 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:55.902935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55a12b-2ae6-4576-8d52-dd9495425329 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:55.926496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 480/960] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 1 19:40:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:56.232177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f394f-0d00-4980-a7ab-81cedefd66aa None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:56.233814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f394f-0d00-4980-a7ab-81cedefd66aa None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:56.233814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f394f-0d00-4980-a7ab-81cedefd66aa None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:56.235086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 481/961] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:56.240266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627cdd9-94e5-499f-8911-11d7ca776620 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:56.240707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627cdd9-94e5-499f-8911-11d7ca776620 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:56.240888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627cdd9-94e5-499f-8911-11d7ca776620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:56.258441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-4627cdd9-94e5-499f-8911-11d7ca776620 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:56.259633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4627cdd9-94e5-499f-8911-11d7ca776620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:56.304558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:56.337507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 481/962] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:56.362503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f394f-0d00-4980-a7ab-81cedefd66aa None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:56.473775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f394f-0d00-4980-a7ab-81cedefd66aa None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:56.512567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df5867-b783-418f-a7b0-fa8acf7b5469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:56.513459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df5867-b783-418f-a7b0-fa8acf7b5469 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:56.514775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df5867-b783-418f-a7b0-fa8acf7b5469 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:56.515200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df5867-b783-418f-a7b0-fa8acf7b5469 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:56.516063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df5867-b783-418f-a7b0-fa8acf7b5469 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:56.517653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df5867-b783-418f-a7b0-fa8acf7b5469 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:56.531826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.sql.core [None req-d5df5867-b783-418f-a7b0-fa8acf7b5469 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 19:40:56.531826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 01 19:40:56.531826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [parameters: {'user_id': '09bff3ce84494bf4b7b1ce8494a3d334', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 01 19:40:56.531826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86779) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-d5df5867-b783-418f-a7b0-fa8acf7b5469 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application conn.query(q) Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result.read() Jul 01 19:40:56.540103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 19:40:56.541125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application next(self.gen) Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application yield resource Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application next(self.gen) Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise self.value Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application session.commit() Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 19:40:56.542295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.session.flush() Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application compat.raise_( Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 19:40:56.543366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 19:40:56.544364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application conn.query(q) Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application result.read() Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 19:40:56.546286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [parameters: {'user_id': '09bff3ce84494bf4b7b1ce8494a3d334', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:56.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 01 19:40:56.547946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 01 19:40:56.548903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 01 19:40:56.548903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 01 19:40:56.548903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 19:40:56.548903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 19:40:56.548903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 01 19:40:56.548903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:56.550948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 482/963] 10.222.0.66 () {60 vars in 1131 bytes} [Fri Jul 1 19:40:56 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:56.562687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627cdd9-94e5-499f-8911-11d7ca776620 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:56.671847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627cdd9-94e5-499f-8911-11d7ca776620 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:56.715974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453dd21a-9b97-4edf-9db4-2b814a761cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:56.717346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dd21a-9b97-4edf-9db4-2b814a761cee None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:56.717945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dd21a-9b97-4edf-9db4-2b814a761cee None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:56.717945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dd21a-9b97-4edf-9db4-2b814a761cee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:56.725068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453dd21a-9b97-4edf-9db4-2b814a761cee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:56.726536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453dd21a-9b97-4edf-9db4-2b814a761cee None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-453dd21a-9b97-4edf-9db4-2b814a761cee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:56.742147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:56.743070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:56.743995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 482/964] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:40:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:56.749278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df5867-b783-418f-a7b0-fa8acf7b5469 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:56.805995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:56.859565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df5867-b783-418f-a7b0-fa8acf7b5469 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:56.896445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ac9c9-1f47-4b18-b6e3-8fd13af36a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:56.897116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ac9c9-1f47-4b18-b6e3-8fd13af36a07 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:56.897396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ac9c9-1f47-4b18-b6e3-8fd13af36a07 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:56.897698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ac9c9-1f47-4b18-b6e3-8fd13af36a07 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:56.898559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ac9c9-1f47-4b18-b6e3-8fd13af36a07 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:56.898915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ac9c9-1f47-4b18-b6e3-8fd13af36a07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:56.900542 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ac9c9-1f47-4b18-b6e3-8fd13af36a07 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:56.940293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 483/965] 10.222.0.66 () {56 vars in 1101 bytes} [Fri Jul 1 19:40:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:57.378490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfb291-aa31-4c17-b0b3-0d2ba117a682 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:57.379041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfb291-aa31-4c17-b0b3-0d2ba117a682 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:57.379041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfb291-aa31-4c17-b0b3-0d2ba117a682 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:57.380694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 483/966] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:40:57.385686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd85ad-ad5a-4750-8afe-3f60bcfa09e0 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:57.386072 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd85ad-ad5a-4750-8afe-3f60bcfa09e0 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:57.386240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd85ad-ad5a-4750-8afe-3f60bcfa09e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:57.403673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-cdbd85ad-ad5a-4750-8afe-3f60bcfa09e0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:57.405197 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-cdbd85ad-ad5a-4750-8afe-3f60bcfa09e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:57.472282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 484/967] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:40:57.496218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfb291-aa31-4c17-b0b3-0d2ba117a682 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:57.529772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:57.604963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfb291-aa31-4c17-b0b3-0d2ba117a682 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:57.640518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5ae95-631e-4193-9f55-0f9a81387b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:57.641248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5ae95-631e-4193-9f55-0f9a81387b60 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:57.641528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5ae95-631e-4193-9f55-0f9a81387b60 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:57.641835 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5ae95-631e-4193-9f55-0f9a81387b60 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:57.653441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5ae95-631e-4193-9f55-0f9a81387b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-18c5ae95-631e-4193-9f55-0f9a81387b60 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:40:57.663030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:57.664002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:40:57.664834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:57.666288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 484/968] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 1 19:40:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:57.671266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd85ad-ad5a-4750-8afe-3f60bcfa09e0 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:57.712110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:57.774163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd85ad-ad5a-4750-8afe-3f60bcfa09e0 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:57.812441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ba264-a2db-4e0d-acb9-3548c77447ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:57.813107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ba264-a2db-4e0d-acb9-3548c77447ce None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:57.813388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ba264-a2db-4e0d-acb9-3548c77447ce None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:57.813686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ba264-a2db-4e0d-acb9-3548c77447ce None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:57.814432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ba264-a2db-4e0d-acb9-3548c77447ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:57.814734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ba264-a2db-4e0d-acb9-3548c77447ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:57.816192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ba264-a2db-4e0d-acb9-3548c77447ce None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:57.829669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 485/969] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 1 19:40:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:57.831183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5ae95-631e-4193-9f55-0f9a81387b60 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:57.925793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:57.935812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5ae95-631e-4193-9f55-0f9a81387b60 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:57.970723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aee38-5065-4fcd-b50d-19419b53f3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:57.971523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aee38-5065-4fcd-b50d-19419b53f3ba None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:57.971808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aee38-5065-4fcd-b50d-19419b53f3ba None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:57.972081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aee38-5065-4fcd-b50d-19419b53f3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:57.983117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aee38-5065-4fcd-b50d-19419b53f3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:57.984570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aee38-5065-4fcd-b50d-19419b53f3ba None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-559aee38-5065-4fcd-b50d-19419b53f3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:58.001760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:58.002661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:58.003542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:58.003542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:58.003542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:58.003542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:58.003760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f117d58-48ba-4984-84bf-c0869783d121 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.003760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f117d58-48ba-4984-84bf-c0869783d121 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.003760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f117d58-48ba-4984-84bf-c0869783d121 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.004930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 486/971] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:58.005660 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 485/971] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:58.008714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f117d58-48ba-4984-84bf-c0869783d121 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:58.009715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b463dd-7ecc-4a16-882a-9e4bcba9a113 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.010007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b463dd-7ecc-4a16-882a-9e4bcba9a113 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.010283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b463dd-7ecc-4a16-882a-9e4bcba9a113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.027174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c0b463dd-7ecc-4a16-882a-9e4bcba9a113 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:58.028345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c0b463dd-7ecc-4a16-882a-9e4bcba9a113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:58.096178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 486/972] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:58.109119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a7048-ec9d-4ded-9bad-975bb0f53210 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.109395 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a7048-ec9d-4ded-9bad-975bb0f53210 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.109696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a7048-ec9d-4ded-9bad-975bb0f53210 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.111212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 487/973] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:40:58.112487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:58.115971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2751749-8d06-4c9c-b1a2-c7022f0a26be None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.116240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2751749-8d06-4c9c-b1a2-c7022f0a26be None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.116506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2751749-8d06-4c9c-b1a2-c7022f0a26be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.117624 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f117d58-48ba-4984-84bf-c0869783d121 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:58.134073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d2751749-8d06-4c9c-b1a2-c7022f0a26be None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:40:58.135253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d2751749-8d06-4c9c-b1a2-c7022f0a26be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:40:58.152797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fbedf-80ac-4bea-bceb-e107cdf6d332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:58.153448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fbedf-80ac-4bea-bceb-e107cdf6d332 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.153755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fbedf-80ac-4bea-bceb-e107cdf6d332 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.154024 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fbedf-80ac-4bea-bceb-e107cdf6d332 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.154912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fbedf-80ac-4bea-bceb-e107cdf6d332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:58.155302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fbedf-80ac-4bea-bceb-e107cdf6d332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:58.156673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fbedf-80ac-4bea-bceb-e107cdf6d332 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:58.168521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 487/974] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:58.169147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fbedf-80ac-4bea-bceb-e107cdf6d332 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:58.204217 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 488/975] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:40:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:40:58.204811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2751749-8d06-4c9c-b1a2-c7022f0a26be None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:58.207739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:58.274965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fbedf-80ac-4bea-bceb-e107cdf6d332 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:58.306347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2751749-8d06-4c9c-b1a2-c7022f0a26be None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:58.308134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97156688-a46f-4166-b905-74f3c2b08811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:58.308786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97156688-a46f-4166-b905-74f3c2b08811 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.309063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97156688-a46f-4166-b905-74f3c2b08811 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.309330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97156688-a46f-4166-b905-74f3c2b08811 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.310073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97156688-a46f-4166-b905-74f3c2b08811 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:58.311996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97156688-a46f-4166-b905-74f3c2b08811 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:58.325058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 488/976] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:40:58 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:40:58.325875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97156688-a46f-4166-b905-74f3c2b08811 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:58.351906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:58.355769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a35612-d052-4d87-acf0-a80dbdfa76b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:58.356611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a35612-d052-4d87-acf0-a80dbdfa76b5 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.356891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a35612-d052-4d87-acf0-a80dbdfa76b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.357165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a35612-d052-4d87-acf0-a80dbdfa76b5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.366289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a35612-d052-4d87-acf0-a80dbdfa76b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:58.367763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a35612-d052-4d87-acf0-a80dbdfa76b5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-e7a35612-d052-4d87-acf0-a80dbdfa76b5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:40:58.385202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:58.386711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:58.387641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 489/977] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 1 19:40:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:58.393080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a35612-d052-4d87-acf0-a80dbdfa76b5 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:58.456305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97156688-a46f-4166-b905-74f3c2b08811 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:58.461546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:58.498859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18064b-423e-44e2-9dd3-9cc4b2fb559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:58.499669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18064b-423e-44e2-9dd3-9cc4b2fb559b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.499951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18064b-423e-44e2-9dd3-9cc4b2fb559b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.500226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18064b-423e-44e2-9dd3-9cc4b2fb559b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.500877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18064b-423e-44e2-9dd3-9cc4b2fb559b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:58.502330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18064b-423e-44e2-9dd3-9cc4b2fb559b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-ce18064b-423e-44e2-9dd3-9cc4b2fb559b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:58.516631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:58.517784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:58.518648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 489/978] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:40:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:58.524030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a35612-d052-4d87-acf0-a80dbdfa76b5 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:58.524193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18064b-423e-44e2-9dd3-9cc4b2fb559b None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:58.563938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5faa-73ec-4836-b1da-160fc9c5d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:58.564643 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5faa-73ec-4836-b1da-160fc9c5d1fd None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.564930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5faa-73ec-4836-b1da-160fc9c5d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.565200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5faa-73ec-4836-b1da-160fc9c5d1fd None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.566075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede5faa-73ec-4836-b1da-160fc9c5d1fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:58.566376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede5faa-73ec-4836-b1da-160fc9c5d1fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:58.567784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede5faa-73ec-4836-b1da-160fc9c5d1fd None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:58.590541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 490/979] 10.222.0.66 () {56 vars in 1149 bytes} [Fri Jul 1 19:40:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:58.591586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5faa-73ec-4836-b1da-160fc9c5d1fd None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:58.624677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:58.643415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18064b-423e-44e2-9dd3-9cc4b2fb559b None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:58.679478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08c67a-2fb2-4b93-8efe-b8a8a21d148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:58.680141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08c67a-2fb2-4b93-8efe-b8a8a21d148f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.680419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08c67a-2fb2-4b93-8efe-b8a8a21d148f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.680708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08c67a-2fb2-4b93-8efe-b8a8a21d148f None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.681396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08c67a-2fb2-4b93-8efe-b8a8a21d148f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:58.681734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08c67a-2fb2-4b93-8efe-b8a8a21d148f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:58.683105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08c67a-2fb2-4b93-8efe-b8a8a21d148f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:58.689200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 490/980] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 1 19:40:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:58.694743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08c67a-2fb2-4b93-8efe-b8a8a21d148f None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:58.698194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5faa-73ec-4836-b1da-160fc9c5d1fd None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:58.717446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:58.736456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23390a6-5548-4059-9e24-35a86e688c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:58.737098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23390a6-5548-4059-9e24-35a86e688c24 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.737409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23390a6-5548-4059-9e24-35a86e688c24 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.737719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23390a6-5548-4059-9e24-35a86e688c24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.802492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08c67a-2fb2-4b93-8efe-b8a8a21d148f None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:58.815040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23390a6-5548-4059-9e24-35a86e688c24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:58.816499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23390a6-5548-4059-9e24-35a86e688c24 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:58.844889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08654-d00c-49ee-8ed6-a259bc06add1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:58.845573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08654-d00c-49ee-8ed6-a259bc06add1 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:58.845922 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08654-d00c-49ee-8ed6-a259bc06add1 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:58.846153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08654-d00c-49ee-8ed6-a259bc06add1 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:58.846980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe08654-d00c-49ee-8ed6-a259bc06add1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:58.847189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe08654-d00c-49ee-8ed6-a259bc06add1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:58.848519 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe08654-d00c-49ee-8ed6-a259bc06add1 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:58.854527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 491/981] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 1 19:40:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:40:58.860447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08654-d00c-49ee-8ed6-a259bc06add1 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:58.878795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:58.923066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 491/982] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:40:58 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 19:40:58.928346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23390a6-5548-4059-9e24-35a86e688c24 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:58.947443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:58.978602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08654-d00c-49ee-8ed6-a259bc06add1 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:59.018747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca185077-262b-497f-9166-0d3f2f43e110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:59.019401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185077-262b-497f-9166-0d3f2f43e110 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:59.019681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185077-262b-497f-9166-0d3f2f43e110 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:59.019955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185077-262b-497f-9166-0d3f2f43e110 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:59.020743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca185077-262b-497f-9166-0d3f2f43e110 None admin] RBAC: Adding query filter params (interface=admin, service_id=175a123f146647639c4c9f3fff0603c8, region_id=RegionOne) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:59.021053 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca185077-262b-497f-9166-0d3f2f43e110 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:59.022453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca185077-262b-497f-9166-0d3f2f43e110 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:59.027608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 492/983] 10.222.0.66 () {56 vars in 1249 bytes} [Fri Jul 1 19:40:58 2022] GET /identity/v3/endpoints?service_id=175a123f146647639c4c9f3fff0603c8&interface=admin®ion_id=RegionOne => generated 193 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:59.033656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca185077-262b-497f-9166-0d3f2f43e110 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:59.040005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23390a6-5548-4059-9e24-35a86e688c24 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:59.077493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee6044-91f9-489e-b26d-df43da9c7667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:59.079339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee6044-91f9-489e-b26d-df43da9c7667 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:59.079989 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee6044-91f9-489e-b26d-df43da9c7667 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:59.079989 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee6044-91f9-489e-b26d-df43da9c7667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:59.091106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee6044-91f9-489e-b26d-df43da9c7667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:59.093135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee6044-91f9-489e-b26d-df43da9c7667 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-57ee6044-91f9-489e-b26d-df43da9c7667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:40:59.112342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:59.113265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:59.115646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:40:59.115646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:40:59.115646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:40:59.115646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:40:59.115646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 492/984] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:40:59.119142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee6044-91f9-489e-b26d-df43da9c7667 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:59.122653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:59.147602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca185077-262b-497f-9166-0d3f2f43e110 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:59.185696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b1323-0ede-4b79-b32d-eeaa613e4898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:59.186533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b1323-0ede-4b79-b32d-eeaa613e4898 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:59.186821 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b1323-0ede-4b79-b32d-eeaa613e4898 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:59.187103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b1323-0ede-4b79-b32d-eeaa613e4898 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:59.187780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b1323-0ede-4b79-b32d-eeaa613e4898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:59.189153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b1323-0ede-4b79-b32d-eeaa613e4898 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:59.195681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 493/985] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 1 19:40:59 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:40:59.196793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:59.235119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee6044-91f9-489e-b26d-df43da9c7667 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:59.272698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3bf7f-dab7-48b1-b937-98447b1a7ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:59.273367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3bf7f-dab7-48b1-b937-98447b1a7ba5 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:59.273674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3bf7f-dab7-48b1-b937-98447b1a7ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:59.273949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3bf7f-dab7-48b1-b937-98447b1a7ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:59.274705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3bf7f-dab7-48b1-b937-98447b1a7ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:59.275010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3bf7f-dab7-48b1-b937-98447b1a7ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:59.276425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3bf7f-dab7-48b1-b937-98447b1a7ba5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:59.290241 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 493/986] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:59.296361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b1323-0ede-4b79-b32d-eeaa613e4898 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:59.409665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b1323-0ede-4b79-b32d-eeaa613e4898 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:59.443716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988dbbd-74a2-4154-a90c-846d7410db00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:59.444511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988dbbd-74a2-4154-a90c-846d7410db00 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:59.444791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988dbbd-74a2-4154-a90c-846d7410db00 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:59.445063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988dbbd-74a2-4154-a90c-846d7410db00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:59.452320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988dbbd-74a2-4154-a90c-846d7410db00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:59.453801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988dbbd-74a2-4154-a90c-846d7410db00 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:59.456456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-d988dbbd-74a2-4154-a90c-846d7410db00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:40:59.472442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:40:59.473406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:40:59.474695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:40:59.474695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:40:59.474695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:40:59.474695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:40:59.474695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:40:59.474695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:40:59.474695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:40:59.474695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:40:59.474695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 494/987] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:40:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:40:59.483804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3bf7f-dab7-48b1-b937-98447b1a7ba5 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:59.590718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3bf7f-dab7-48b1-b937-98447b1a7ba5 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:59.619963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:40:59.627678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237a0ad-af72-46ca-ab8c-52ca3ce4244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:59.628345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237a0ad-af72-46ca-ab8c-52ca3ce4244e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:59.628626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237a0ad-af72-46ca-ab8c-52ca3ce4244e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:59.628898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237a0ad-af72-46ca-ab8c-52ca3ce4244e None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:59.629708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237a0ad-af72-46ca-ab8c-52ca3ce4244e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:59.630030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237a0ad-af72-46ca-ab8c-52ca3ce4244e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:59.631450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237a0ad-af72-46ca-ab8c-52ca3ce4244e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:59.649319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 494/988] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:40:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:40:59.650745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988dbbd-74a2-4154-a90c-846d7410db00 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:40:59.754084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988dbbd-74a2-4154-a90c-846d7410db00 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:40:59.788323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c2e3d-286f-46bb-ba9e-4ba788d719fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:40:59.788973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c2e3d-286f-46bb-ba9e-4ba788d719fa None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:40:59.789253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c2e3d-286f-46bb-ba9e-4ba788d719fa None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:40:59.789527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c2e3d-286f-46bb-ba9e-4ba788d719fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:40:59.790376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c2e3d-286f-46bb-ba9e-4ba788d719fa None admin] RBAC: Adding query filter params (role.id=0f21ddda19344dfd855c441dd59142ef, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=cfa25a63dd6f4ad08b0f858356528284) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:40:59.790684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c2e3d-286f-46bb-ba9e-4ba788d719fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:40:59.792111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c2e3d-286f-46bb-ba9e-4ba788d719fa None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:40:59.797949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 495/989] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:40:59 2022] GET /identity/v3/role_assignments?user.id=cfa25a63dd6f4ad08b0f858356528284&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0f21ddda19344dfd855c441dd59142ef => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:00.414512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cfe1f-0553-41b2-adcf-d9265a0b0c7d None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:00.414925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cfe1f-0553-41b2-adcf-d9265a0b0c7d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:00.415113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cfe1f-0553-41b2-adcf-d9265a0b0c7d None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:00.416875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 495/990] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:00.421949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae8207-b86c-4feb-972c-9166ef5bb9d9 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:00.422237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae8207-b86c-4feb-972c-9166ef5bb9d9 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:00.422537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae8207-b86c-4feb-972c-9166ef5bb9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:00.440101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-34ae8207-b86c-4feb-972c-9166ef5bb9d9 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:00.441316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-34ae8207-b86c-4feb-972c-9166ef5bb9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:00.505290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:00.515820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 496/991] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:41:00.539177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cfe1f-0553-41b2-adcf-d9265a0b0c7d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:00.642001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cfe1f-0553-41b2-adcf-d9265a0b0c7d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:00.675325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f743a-8387-4ae5-b739-250e6cfbd159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:00.676072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f743a-8387-4ae5-b739-250e6cfbd159 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:00.676356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f743a-8387-4ae5-b739-250e6cfbd159 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:00.676636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f743a-8387-4ae5-b739-250e6cfbd159 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:00.677285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f743a-8387-4ae5-b739-250e6cfbd159 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:00.678721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f743a-8387-4ae5-b739-250e6cfbd159 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-b25f743a-8387-4ae5-b739-250e6cfbd159 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:00.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:00.692092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:00.693124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 496/992] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:41:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:00.700051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae8207-b86c-4feb-972c-9166ef5bb9d9 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:00.804504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae8207-b86c-4feb-972c-9166ef5bb9d9 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:00.839409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fc0f0-b530-4173-92ff-3577903d03ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:00.840072 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fc0f0-b530-4173-92ff-3577903d03ad None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:00.840353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fc0f0-b530-4173-92ff-3577903d03ad None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:00.840627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fc0f0-b530-4173-92ff-3577903d03ad None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:00.841330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fc0f0-b530-4173-92ff-3577903d03ad None admin] RBAC: Adding query filter params (name=compute) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:00.841765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fc0f0-b530-4173-92ff-3577903d03ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:00.843141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fc0f0-b530-4173-92ff-3577903d03ad None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:00.852564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 497/993] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:41:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:00.853803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f743a-8387-4ae5-b739-250e6cfbd159 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:00.912357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:00.961435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f743a-8387-4ae5-b739-250e6cfbd159 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:00.995061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15de056-d665-426c-8b99-a4da554796bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:00.995746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15de056-d665-426c-8b99-a4da554796bd None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:00.996031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15de056-d665-426c-8b99-a4da554796bd None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:00.996305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15de056-d665-426c-8b99-a4da554796bd None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:00.997006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15de056-d665-426c-8b99-a4da554796bd None admin] RBAC: Adding query filter params (type=compute) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:00.997309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15de056-d665-426c-8b99-a4da554796bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:00.999270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15de056-d665-426c-8b99-a4da554796bd None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:01.009035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 497/994] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:41:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:01.010348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fc0f0-b530-4173-92ff-3577903d03ad None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:01.070110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:01.116129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fc0f0-b530-4173-92ff-3577903d03ad None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:01.149480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b3dd7-4b29-4daf-9fc5-f30968bb4928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:01.150152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b3dd7-4b29-4daf-9fc5-f30968bb4928 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:01.150433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b3dd7-4b29-4daf-9fc5-f30968bb4928 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:01.150702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b3dd7-4b29-4daf-9fc5-f30968bb4928 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:01.151475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b3dd7-4b29-4daf-9fc5-f30968bb4928 None admin] RBAC: Adding query filter params (interface=public, service_id=8bee8969f5854b59bf241e2d24a03b49, region_id=RegionOne) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:01.151781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b3dd7-4b29-4daf-9fc5-f30968bb4928 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:01.153173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b3dd7-4b29-4daf-9fc5-f30968bb4928 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:01.159132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 498/995] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 1 19:41:01 2022] GET /identity/v3/endpoints?service_id=8bee8969f5854b59bf241e2d24a03b49&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:01.164667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15de056-d665-426c-8b99-a4da554796bd None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:01.266854 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15de056-d665-426c-8b99-a4da554796bd None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:01.283945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:01.303054 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2b147-4c61-49f9-b296-9430739794a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:01.303725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2b147-4c61-49f9-b296-9430739794a0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:01.304007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2b147-4c61-49f9-b296-9430739794a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:01.304283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2b147-4c61-49f9-b296-9430739794a0 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:01.304967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2b147-4c61-49f9-b296-9430739794a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:01.306388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2b147-4c61-49f9-b296-9430739794a0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:01.321632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 498/996] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 1 19:41:01 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:41:01.337652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f414db2-d533-499f-82aa-6ae5fd515b7b None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:01.337933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f414db2-d533-499f-82aa-6ae5fd515b7b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:01.338215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f414db2-d533-499f-82aa-6ae5fd515b7b None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:01.339671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 499/997] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:01.346412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e65932-d5bf-44db-af9f-1ee709ae5400 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:01.346706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e65932-d5bf-44db-af9f-1ee709ae5400 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:01.346986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e65932-d5bf-44db-af9f-1ee709ae5400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:01.365198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-02e65932-d5bf-44db-af9f-1ee709ae5400 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:01.366459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-02e65932-d5bf-44db-af9f-1ee709ae5400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:01.445733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 499/998] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:01.468973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f414db2-d533-499f-82aa-6ae5fd515b7b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:01.585448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f414db2-d533-499f-82aa-6ae5fd515b7b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:01.609353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:01.621749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817b47-4345-4ccf-a0dc-6048c7f9b39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:01.622557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817b47-4345-4ccf-a0dc-6048c7f9b39c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:01.622875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817b47-4345-4ccf-a0dc-6048c7f9b39c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:01.623215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2817b47-4345-4ccf-a0dc-6048c7f9b39c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:01.623802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2817b47-4345-4ccf-a0dc-6048c7f9b39c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:01.625183 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2817b47-4345-4ccf-a0dc-6048c7f9b39c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-f2817b47-4345-4ccf-a0dc-6048c7f9b39c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:01.637538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:01.639342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:01.645738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 500/999] 10.222.0.66 () {56 vars in 1126 bytes} [Fri Jul 1 19:41:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:01.648133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e65932-d5bf-44db-af9f-1ee709ae5400 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:01.748016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e65932-d5bf-44db-af9f-1ee709ae5400 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:01.761287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:01.784406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d671350-881f-459c-a726-0cc23354430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:01.785345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d671350-881f-459c-a726-0cc23354430e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:01.785650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d671350-881f-459c-a726-0cc23354430e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:01.785928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d671350-881f-459c-a726-0cc23354430e None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:01.786636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d671350-881f-459c-a726-0cc23354430e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:01.786941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d671350-881f-459c-a726-0cc23354430e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:01.788316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d671350-881f-459c-a726-0cc23354430e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:01.798024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 500/1000] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 1 19:41:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:01.801033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817b47-4345-4ccf-a0dc-6048c7f9b39c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:01.881565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb9417-3b70-4a0f-927c-1af3008c1c8d None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:01.881924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb9417-3b70-4a0f-927c-1af3008c1c8d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:01.882148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb9417-3b70-4a0f-927c-1af3008c1c8d None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:01.883645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 501/1001] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:01.888252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a7386-d831-46bc-9f80-3c574cd86beb None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:01.889074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a7386-d831-46bc-9f80-3c574cd86beb None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:01.889355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a7386-d831-46bc-9f80-3c574cd86beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:01.906324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a59a7386-d831-46bc-9f80-3c574cd86beb None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:01.907508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a59a7386-d831-46bc-9f80-3c574cd86beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:01.909227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2817b47-4345-4ccf-a0dc-6048c7f9b39c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:01.950389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376c3fa-39d8-4423-9821-23bc0e89a283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:01.951451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376c3fa-39d8-4423-9821-23bc0e89a283 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:01.951736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376c3fa-39d8-4423-9821-23bc0e89a283 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:01.952011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376c3fa-39d8-4423-9821-23bc0e89a283 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:01.953046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376c3fa-39d8-4423-9821-23bc0e89a283 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:01.953355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376c3fa-39d8-4423-9821-23bc0e89a283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:01.954785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376c3fa-39d8-4423-9821-23bc0e89a283 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:01.960476 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 501/1002] 10.222.0.66 () {56 vars in 1122 bytes} [Fri Jul 1 19:41:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:01.967029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376c3fa-39d8-4423-9821-23bc0e89a283 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:01.977279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 502/1003] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:41:02.001026 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a7386-d831-46bc-9f80-3c574cd86beb None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:02.021948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:02.083628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376c3fa-39d8-4423-9821-23bc0e89a283 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:02.107976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a7386-d831-46bc-9f80-3c574cd86beb None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:02.116320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:02.124181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bd308-9a2b-4ef3-8b4e-806cb27b9d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:02.124850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bd308-9a2b-4ef3-8b4e-806cb27b9d76 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:02.125132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bd308-9a2b-4ef3-8b4e-806cb27b9d76 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:02.125405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bd308-9a2b-4ef3-8b4e-806cb27b9d76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:02.126179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bd308-9a2b-4ef3-8b4e-806cb27b9d76 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:02.127586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bd308-9a2b-4ef3-8b4e-806cb27b9d76 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:02.146002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 502/1004] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 1 19:41:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:41:02.163409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62740148-6b9a-48de-8852-1721e95ac4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:02.164320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62740148-6b9a-48de-8852-1721e95ac4ae None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:02.164605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62740148-6b9a-48de-8852-1721e95ac4ae None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:02.164879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62740148-6b9a-48de-8852-1721e95ac4ae None admin] PATH_INFO: `/v3/services/image` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:02.165547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62740148-6b9a-48de-8852-1721e95ac4ae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:02.167027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62740148-6b9a-48de-8852-1721e95ac4ae None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-62740148-6b9a-48de-8852-1721e95ac4ae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:02.180656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:02.182047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 19:41:02.182903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:02.186256 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 503/1005] 10.222.0.66 () {56 vars in 1105 bytes} [Fri Jul 1 19:41:02 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:02.188518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bd308-9a2b-4ef3-8b4e-806cb27b9d76 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:02.300181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bd308-9a2b-4ef3-8b4e-806cb27b9d76 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:02.312318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:02.341143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68811785-fe8f-4448-8450-4d51bed86990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:02.342908 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68811785-fe8f-4448-8450-4d51bed86990 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:02.343453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68811785-fe8f-4448-8450-4d51bed86990 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:02.343736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68811785-fe8f-4448-8450-4d51bed86990 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:02.344452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68811785-fe8f-4448-8450-4d51bed86990 None admin] RBAC: Adding query filter params (name=image) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:02.344762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68811785-fe8f-4448-8450-4d51bed86990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:02.346175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68811785-fe8f-4448-8450-4d51bed86990 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:02.351299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 503/1006] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:02.357023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62740148-6b9a-48de-8852-1721e95ac4ae None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:02.469202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62740148-6b9a-48de-8852-1721e95ac4ae None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:02.506703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189274b-885a-4473-ad0b-695a8117ce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:02.507376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189274b-885a-4473-ad0b-695a8117ce2b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:02.507653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189274b-885a-4473-ad0b-695a8117ce2b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:02.507924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189274b-885a-4473-ad0b-695a8117ce2b None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:02.508626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189274b-885a-4473-ad0b-695a8117ce2b None admin] RBAC: Adding query filter params (type=image) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:02.508924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189274b-885a-4473-ad0b-695a8117ce2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:02.510349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189274b-885a-4473-ad0b-695a8117ce2b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:02.516483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 504/1007] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:03.395432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cc6bc-7633-411c-8ba2-779ad135cf2a None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:03.395951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cc6bc-7633-411c-8ba2-779ad135cf2a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:03.395951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cc6bc-7633-411c-8ba2-779ad135cf2a None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:03.397494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 504/1008] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:03.402719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9518ef-12a2-4ada-a52e-0c0ececeddec None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:03.403108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9518ef-12a2-4ada-a52e-0c0ececeddec None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:03.403288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9518ef-12a2-4ada-a52e-0c0ececeddec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:03.420312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5e9518ef-12a2-4ada-a52e-0c0ececeddec None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:03.421479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5e9518ef-12a2-4ada-a52e-0c0ececeddec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:03.446082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:03.491254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 505/1009] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:03 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:03.515392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cc6bc-7633-411c-8ba2-779ad135cf2a None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:03.633124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cc6bc-7633-411c-8ba2-779ad135cf2a None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:03.670531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3b09d-5828-4eeb-b8ba-7b374d41f26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:03.671327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3b09d-5828-4eeb-b8ba-7b374d41f26c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:03.671610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3b09d-5828-4eeb-b8ba-7b374d41f26c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:03.671912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3b09d-5828-4eeb-b8ba-7b374d41f26c None admin] PATH_INFO: `/v3/services/compute` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:03.672559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3b09d-5828-4eeb-b8ba-7b374d41f26c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:03.674099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3b09d-5828-4eeb-b8ba-7b374d41f26c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-4ce3b09d-5828-4eeb-b8ba-7b374d41f26c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:03.686856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:03.687881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 19:41:03.688718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:03.690279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 505/1010] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:41:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:03.694043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9518ef-12a2-4ada-a52e-0c0ececeddec None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:03.798532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9518ef-12a2-4ada-a52e-0c0ececeddec None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:03.833006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef12d8-2978-4e33-af6f-c367682446e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:03.833969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef12d8-2978-4e33-af6f-c367682446e8 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:03.834461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef12d8-2978-4e33-af6f-c367682446e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:03.834918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef12d8-2978-4e33-af6f-c367682446e8 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:03.835796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef12d8-2978-4e33-af6f-c367682446e8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:03.836283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef12d8-2978-4e33-af6f-c367682446e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:03.837865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef12d8-2978-4e33-af6f-c367682446e8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:03.843028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 506/1011] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:41:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:03.848829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3b09d-5828-4eeb-b8ba-7b374d41f26c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:03.890544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:03.952615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3b09d-5828-4eeb-b8ba-7b374d41f26c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:03.987756 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c762acf-fa60-4743-87a6-537af786c209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:03.988463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c762acf-fa60-4743-87a6-537af786c209 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:03.988744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c762acf-fa60-4743-87a6-537af786c209 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:03.989015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c762acf-fa60-4743-87a6-537af786c209 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:03.989806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c762acf-fa60-4743-87a6-537af786c209 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:03.990112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c762acf-fa60-4743-87a6-537af786c209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:03.991479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c762acf-fa60-4743-87a6-537af786c209 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:03.998576 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 506/1012] 10.222.0.66 () {56 vars in 1112 bytes} [Fri Jul 1 19:41:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:04.002790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef12d8-2978-4e33-af6f-c367682446e8 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:04.028137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:04.104648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef12d8-2978-4e33-af6f-c367682446e8 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:04.139426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02ab06-5893-49e4-ab76-a8586a187da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:04.140141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02ab06-5893-49e4-ab76-a8586a187da5 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:04.140422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02ab06-5893-49e4-ab76-a8586a187da5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:04.140695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02ab06-5893-49e4-ab76-a8586a187da5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:04.141492 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02ab06-5893-49e4-ab76-a8586a187da5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:04.142899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02ab06-5893-49e4-ab76-a8586a187da5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:04.162714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 507/1013] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 1 19:41:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:41:04.269514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614442-d6d7-4bbd-917c-c7f46b6702bf None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:04.269839 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614442-d6d7-4bbd-917c-c7f46b6702bf None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:04.270015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614442-d6d7-4bbd-917c-c7f46b6702bf None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:04.271650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 507/1014] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:04.277106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec2061-9217-4a8f-87dc-34b9d4c25ffd None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:04.277399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec2061-9217-4a8f-87dc-34b9d4c25ffd None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:04.277720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec2061-9217-4a8f-87dc-34b9d4c25ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:04.297895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-3aec2061-9217-4a8f-87dc-34b9d4c25ffd None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:04.299118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3aec2061-9217-4a8f-87dc-34b9d4c25ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:04.374783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 508/1015] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:04 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:04.399198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614442-d6d7-4bbd-917c-c7f46b6702bf None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:04.511372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614442-d6d7-4bbd-917c-c7f46b6702bf None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:04.513179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:04.552304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5be15-689a-4c96-846e-87a903b896fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:04.553102 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5be15-689a-4c96-846e-87a903b896fe None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:04.553379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5be15-689a-4c96-846e-87a903b896fe None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:04.553680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5be15-689a-4c96-846e-87a903b896fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:04.554860 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5be15-689a-4c96-846e-87a903b896fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:04.556304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5be15-689a-4c96-846e-87a903b896fe None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:04.558149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 508/1016] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:04.563694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec2061-9217-4a8f-87dc-34b9d4c25ffd None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:04.572124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:04.609507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a491a6-9a26-4537-97e8-8c4ede4fe19f None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:04.610295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a491a6-9a26-4537-97e8-8c4ede4fe19f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:04.610577 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a491a6-9a26-4537-97e8-8c4ede4fe19f None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:04.612579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 509/1017] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:41:04.617557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f5bf9-bc09-48a9-b333-f38415b56142 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:04.617892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f5bf9-bc09-48a9-b333-f38415b56142 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:04.618172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f5bf9-bc09-48a9-b333-f38415b56142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:04.635536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d05f5bf9-bc09-48a9-b333-f38415b56142 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:04.636849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d05f5bf9-bc09-48a9-b333-f38415b56142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:04.673702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec2061-9217-4a8f-87dc-34b9d4c25ffd None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:04.709100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7b9cf-6d97-4f4b-ab35-c908303c0efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:04.709868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7b9cf-6d97-4f4b-ab35-c908303c0efc None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:04.710146 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7b9cf-6d97-4f4b-ab35-c908303c0efc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:04.710420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7b9cf-6d97-4f4b-ab35-c908303c0efc None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:04.711136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7b9cf-6d97-4f4b-ab35-c908303c0efc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:04.712570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7b9cf-6d97-4f4b-ab35-c908303c0efc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:04.725740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 510/1018] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:04 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:04.745614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 509/1019] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:41:04 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:41:04.746370 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f5bf9-bc09-48a9-b333-f38415b56142 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:04.870693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:04.875307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f5bf9-bc09-48a9-b333-f38415b56142 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:04.915166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30d28b-d13a-4ee1-99c5-b3afd58a110f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:04.915835 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d28b-d13a-4ee1-99c5-b3afd58a110f None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:04.916118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d28b-d13a-4ee1-99c5-b3afd58a110f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:04.916395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d28b-d13a-4ee1-99c5-b3afd58a110f None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:04.917125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30d28b-d13a-4ee1-99c5-b3afd58a110f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:04.918587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30d28b-d13a-4ee1-99c5-b3afd58a110f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:04.933881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 511/1020] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:41:04 2022] POST /identity/v3/services => generated 242 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:41:06.383114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c752-493a-41ed-a2db-10b2555f8f76 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:06.383633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c752-493a-41ed-a2db-10b2555f8f76 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:06.383633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c752-493a-41ed-a2db-10b2555f8f76 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:06.385013 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 510/1021] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:06.390905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5810054-b994-4361-a824-7513500056d3 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:06.391319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5810054-b994-4361-a824-7513500056d3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:06.391510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5810054-b994-4361-a824-7513500056d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:06.409090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a5810054-b994-4361-a824-7513500056d3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:06.410571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a5810054-b994-4361-a824-7513500056d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:06.485627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 512/1022] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:06.509148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9c752-493a-41ed-a2db-10b2555f8f76 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:06.575893 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:06.622349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9c752-493a-41ed-a2db-10b2555f8f76 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:06.657245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bb349-c02b-48d9-8a5c-b0feacf1e719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:06.658003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bb349-c02b-48d9-8a5c-b0feacf1e719 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:06.658280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bb349-c02b-48d9-8a5c-b0feacf1e719 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:06.658548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bb349-c02b-48d9-8a5c-b0feacf1e719 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:06.668926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bb349-c02b-48d9-8a5c-b0feacf1e719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-ef1bb349-c02b-48d9-8a5c-b0feacf1e719 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:06.678394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:06.679345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:41:06.680183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:06.681947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 511/1023] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 1 19:41:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:06.685980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5810054-b994-4361-a824-7513500056d3 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:06.771933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:06.789152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5810054-b994-4361-a824-7513500056d3 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:06.822874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b6188-0a5f-4caa-ac29-d173a907345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:06.823533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b6188-0a5f-4caa-ac29-d173a907345b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:06.823808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b6188-0a5f-4caa-ac29-d173a907345b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:06.824077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b6188-0a5f-4caa-ac29-d173a907345b None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:06.824805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b6188-0a5f-4caa-ac29-d173a907345b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:06.825104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b6188-0a5f-4caa-ac29-d173a907345b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:06.827434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b6188-0a5f-4caa-ac29-d173a907345b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:06.835653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 513/1024] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 1 19:41:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:06.841033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bb349-c02b-48d9-8a5c-b0feacf1e719 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:06.848968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11699a-5f6a-485a-aa96-e3f986083cf1 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:06.849263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11699a-5f6a-485a-aa96-e3f986083cf1 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:06.849526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11699a-5f6a-485a-aa96-e3f986083cf1 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:06.851115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 514/1025] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:41:06.856324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed9d1c-77d0-440d-8dd6-2b216f725c68 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:06.856672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed9d1c-77d0-440d-8dd6-2b216f725c68 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:06.857048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed9d1c-77d0-440d-8dd6-2b216f725c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:06.874567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-59ed9d1c-77d0-440d-8dd6-2b216f725c68 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:06.875864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-59ed9d1c-77d0-440d-8dd6-2b216f725c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:06.950507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 515/1026] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:41:06.955065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bb349-c02b-48d9-8a5c-b0feacf1e719 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:06.972728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:06.977101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed9d1c-77d0-440d-8dd6-2b216f725c68 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:06.991715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c30052-2468-49a3-94e8-164dc4c83449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:06.992507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30052-2468-49a3-94e8-164dc4c83449 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:06.992784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30052-2468-49a3-94e8-164dc4c83449 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:06.993051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30052-2468-49a3-94e8-164dc4c83449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.004092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c30052-2468-49a3-94e8-164dc4c83449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:07.005533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c30052-2468-49a3-94e8-164dc4c83449 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-d0c30052-2468-49a3-94e8-164dc4c83449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:41:07.022350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:07.023264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:07.024173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:41:07.024173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:41:07.024173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:07.024173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:07.024173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 512/1027] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:07.024715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ef122-6a19-49a9-8b59-e66a1c538e48 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.025012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ef122-6a19-49a9-8b59-e66a1c538e48 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.025268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ef122-6a19-49a9-8b59-e66a1c538e48 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.026544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 513/1028] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:07.029314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ef122-6a19-49a9-8b59-e66a1c538e48 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:07.059249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:07.083237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed9d1c-77d0-440d-8dd6-2b216f725c68 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:07.117743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2cc78-e38c-48cc-946f-7344aa1a3af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:07.118532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2cc78-e38c-48cc-946f-7344aa1a3af7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.118815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2cc78-e38c-48cc-946f-7344aa1a3af7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.119087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2cc78-e38c-48cc-946f-7344aa1a3af7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.120264 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2cc78-e38c-48cc-946f-7344aa1a3af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:07.121733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2cc78-e38c-48cc-946f-7344aa1a3af7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:07.123707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 516/1029] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:07.124940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f4184-cc14-4611-a19d-2117b2965854 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.125225 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f4184-cc14-4611-a19d-2117b2965854 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.125503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f4184-cc14-4611-a19d-2117b2965854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.131170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ef122-6a19-49a9-8b59-e66a1c538e48 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:07.142346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-bf0f4184-cc14-4611-a19d-2117b2965854 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:07.144333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-bf0f4184-cc14-4611-a19d-2117b2965854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:07.168797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa722f02-e06b-4665-a6fe-a4192f13b9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:07.170136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa722f02-e06b-4665-a6fe-a4192f13b9b6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.170419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa722f02-e06b-4665-a6fe-a4192f13b9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.170693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa722f02-e06b-4665-a6fe-a4192f13b9b6 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.171411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa722f02-e06b-4665-a6fe-a4192f13b9b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:07.171713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa722f02-e06b-4665-a6fe-a4192f13b9b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:07.173075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa722f02-e06b-4665-a6fe-a4192f13b9b6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:07.184379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 514/1030] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:07.185205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa722f02-e06b-4665-a6fe-a4192f13b9b6 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:07.215308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 517/1031] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:07.216072 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f4184-cc14-4611-a19d-2117b2965854 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:07.219632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:07.291908 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:07.293315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa722f02-e06b-4665-a6fe-a4192f13b9b6 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:07.315965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f4184-cc14-4611-a19d-2117b2965854 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:07.327839 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33432df8-9066-4acc-a11e-c0c503003a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:07.328503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33432df8-9066-4acc-a11e-c0c503003a4c None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.328782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33432df8-9066-4acc-a11e-c0c503003a4c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.329059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33432df8-9066-4acc-a11e-c0c503003a4c None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.329795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33432df8-9066-4acc-a11e-c0c503003a4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:07.331236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33432df8-9066-4acc-a11e-c0c503003a4c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:07.350838 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 515/1032] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:41:07 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:41:07.351002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce88df-5f29-49ab-b6e8-d00fc5a3cdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:07.351631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33432df8-9066-4acc-a11e-c0c503003a4c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:07.351958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce88df-5f29-49ab-b6e8-d00fc5a3cdaa None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.352252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce88df-5f29-49ab-b6e8-d00fc5a3cdaa None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.352591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce88df-5f29-49ab-b6e8-d00fc5a3cdaa None admin] PATH_INFO: `/v3/users/nova` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.360767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce88df-5f29-49ab-b6e8-d00fc5a3cdaa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:07.362382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce88df-5f29-49ab-b6e8-d00fc5a3cdaa None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-cdce88df-5f29-49ab-b6e8-d00fc5a3cdaa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:41:07.379736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:07.380776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:07.381790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 518/1033] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 1 19:41:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:07.387324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce88df-5f29-49ab-b6e8-d00fc5a3cdaa None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:07.462862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:07.483704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33432df8-9066-4acc-a11e-c0c503003a4c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:07.505006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce88df-5f29-49ab-b6e8-d00fc5a3cdaa None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:07.526280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c4f45-5329-417a-8ed2-9af0ec02f4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:07.527081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4f45-5329-417a-8ed2-9af0ec02f4db None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.527364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4f45-5329-417a-8ed2-9af0ec02f4db None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.527635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4f45-5329-417a-8ed2-9af0ec02f4db None admin] PATH_INFO: `/v3/services/image` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.528291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c4f45-5329-417a-8ed2-9af0ec02f4db None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:07.530580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c4f45-5329-417a-8ed2-9af0ec02f4db None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-358c4f45-5329-417a-8ed2-9af0ec02f4db None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:07.542939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:07.543922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:07.545364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 516/1034] 10.222.0.66 () {56 vars in 1105 bytes} [Fri Jul 1 19:41:07 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:07.551274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c4f45-5329-417a-8ed2-9af0ec02f4db None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:07.552591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ea9f8-749c-463d-850f-18c397de1b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:07.553273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ea9f8-749c-463d-850f-18c397de1b08 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.553549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ea9f8-749c-463d-850f-18c397de1b08 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.553852 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ea9f8-749c-463d-850f-18c397de1b08 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.554694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ea9f8-749c-463d-850f-18c397de1b08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:07.554994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ea9f8-749c-463d-850f-18c397de1b08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:07.556423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ea9f8-749c-463d-850f-18c397de1b08 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:07.559693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:07.582854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 519/1035] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 1 19:41:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:07.588862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ea9f8-749c-463d-850f-18c397de1b08 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:07.668514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c4f45-5329-417a-8ed2-9af0ec02f4db None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:07.691588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ea9f8-749c-463d-850f-18c397de1b08 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:07.706678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075cf9f-63f0-4d8c-8043-5f854aae31b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:07.707678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075cf9f-63f0-4d8c-8043-5f854aae31b6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.708161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075cf9f-63f0-4d8c-8043-5f854aae31b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.708624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075cf9f-63f0-4d8c-8043-5f854aae31b6 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.709656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075cf9f-63f0-4d8c-8043-5f854aae31b6 None admin] RBAC: Adding query filter params (name=image) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:07.710267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075cf9f-63f0-4d8c-8043-5f854aae31b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:07.711714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075cf9f-63f0-4d8c-8043-5f854aae31b6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:07.718162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 517/1036] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:07.726730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075cf9f-63f0-4d8c-8043-5f854aae31b6 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:07.729049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c668521-506e-424d-aa44-0a579fd09a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:07.729915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c668521-506e-424d-aa44-0a579fd09a02 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.730352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c668521-506e-424d-aa44-0a579fd09a02 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.730633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c668521-506e-424d-aa44-0a579fd09a02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.744181 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:07.799858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:07.811240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c668521-506e-424d-aa44-0a579fd09a02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:07.813094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c668521-506e-424d-aa44-0a579fd09a02 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:07.834495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075cf9f-63f0-4d8c-8043-5f854aae31b6 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:07.872450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf3253-f93e-4597-90c0-bbce681c9680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:07.873224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf3253-f93e-4597-90c0-bbce681c9680 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:07.873396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf3253-f93e-4597-90c0-bbce681c9680 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:07.873707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf3253-f93e-4597-90c0-bbce681c9680 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:07.874439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf3253-f93e-4597-90c0-bbce681c9680 None admin] RBAC: Adding query filter params (type=image) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:07.874920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf3253-f93e-4597-90c0-bbce681c9680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:07.876116 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf3253-f93e-4597-90c0-bbce681c9680 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:07.881326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 518/1037] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:07.887083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf3253-f93e-4597-90c0-bbce681c9680 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:07.926258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 520/1038] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 19:41:07.931985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c668521-506e-424d-aa44-0a579fd09a02 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:07.976373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:08.002322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf3253-f93e-4597-90c0-bbce681c9680 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:08.028215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:08.036634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c668521-506e-424d-aa44-0a579fd09a02 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:08.043107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144ad4d-a1d1-4e38-8cec-2aa4f8a149ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:08.043766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144ad4d-a1d1-4e38-8cec-2aa4f8a149ad None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:08.044045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144ad4d-a1d1-4e38-8cec-2aa4f8a149ad None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:08.045454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144ad4d-a1d1-4e38-8cec-2aa4f8a149ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:08.046283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144ad4d-a1d1-4e38-8cec-2aa4f8a149ad None admin] RBAC: Adding query filter params (interface=public, service_id=35f6c2b1cb694662b6d0fa1caee43a21, region_id=RegionOne) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:08.046592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144ad4d-a1d1-4e38-8cec-2aa4f8a149ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:08.048428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144ad4d-a1d1-4e38-8cec-2aa4f8a149ad None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:08.055132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 519/1039] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 1 19:41:07 2022] GET /identity/v3/endpoints?service_id=35f6c2b1cb694662b6d0fa1caee43a21&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:08.061126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144ad4d-a1d1-4e38-8cec-2aa4f8a149ad None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:08.077677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12bfdc-cc5e-4346-a9c4-b901fe84ffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:08.078472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12bfdc-cc5e-4346-a9c4-b901fe84ffad None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:08.078750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12bfdc-cc5e-4346-a9c4-b901fe84ffad None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:08.079022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12bfdc-cc5e-4346-a9c4-b901fe84ffad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:08.089271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12bfdc-cc5e-4346-a9c4-b901fe84ffad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:08.090761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12bfdc-cc5e-4346-a9c4-b901fe84ffad None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-8f12bfdc-cc5e-4346-a9c4-b901fe84ffad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:41:08.108567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:08.109573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:08.110501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:41:08.110501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:41:08.110501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:08.110501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:08.110501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 521/1040] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:08.115461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12bfdc-cc5e-4346-a9c4-b901fe84ffad None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:08.169569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144ad4d-a1d1-4e38-8cec-2aa4f8a149ad None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:08.205815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382f9f6-44ef-4ec5-ba91-93367ebb7b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:08.206472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382f9f6-44ef-4ec5-ba91-93367ebb7b5b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:08.206745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382f9f6-44ef-4ec5-ba91-93367ebb7b5b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:08.207014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382f9f6-44ef-4ec5-ba91-93367ebb7b5b None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:08.207680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382f9f6-44ef-4ec5-ba91-93367ebb7b5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:08.209039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382f9f6-44ef-4ec5-ba91-93367ebb7b5b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:08.216967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 520/1041] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 1 19:41:08 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:41:08.223716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12bfdc-cc5e-4346-a9c4-b901fe84ffad None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:08.264862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9062546-9d28-450f-92ea-57178adeb682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:08.265473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9062546-9d28-450f-92ea-57178adeb682 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:08.265788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9062546-9d28-450f-92ea-57178adeb682 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:08.266066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9062546-9d28-450f-92ea-57178adeb682 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:08.266816 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9062546-9d28-450f-92ea-57178adeb682 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:08.267124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9062546-9d28-450f-92ea-57178adeb682 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:08.268565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9062546-9d28-450f-92ea-57178adeb682 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:08.286563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 522/1042] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:08.288139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382f9f6-44ef-4ec5-ba91-93367ebb7b5b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:08.300218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:08.402564 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382f9f6-44ef-4ec5-ba91-93367ebb7b5b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:08.438239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dbedc-e1fc-46e9-91db-2847d2e79645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:08.439037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dbedc-e1fc-46e9-91db-2847d2e79645 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:08.439319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dbedc-e1fc-46e9-91db-2847d2e79645 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:08.439595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dbedc-e1fc-46e9-91db-2847d2e79645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:08.446486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257dbedc-e1fc-46e9-91db-2847d2e79645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:08.447926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257dbedc-e1fc-46e9-91db-2847d2e79645 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-257dbedc-e1fc-46e9-91db-2847d2e79645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:41:08.463030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:08.464385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:08.465275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:41:08.465275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:41:08.465275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:41:08.465275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:41:08.465275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:41:08.465275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:41:08.465275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:41:08.465275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:08.465275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 521/1043] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:41:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:08.472267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9062546-9d28-450f-92ea-57178adeb682 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:08.482600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:08.586842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9062546-9d28-450f-92ea-57178adeb682 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:08.624619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fde5f-c0c1-487d-b1bf-2b0b5c05440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:08.625283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fde5f-c0c1-487d-b1bf-2b0b5c05440a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:08.625564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fde5f-c0c1-487d-b1bf-2b0b5c05440a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:08.626022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fde5f-c0c1-487d-b1bf-2b0b5c05440a None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:08.626810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fde5f-c0c1-487d-b1bf-2b0b5c05440a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:08.627118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fde5f-c0c1-487d-b1bf-2b0b5c05440a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:08.628556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fde5f-c0c1-487d-b1bf-2b0b5c05440a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:08.646320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 523/1044] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:41:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:08.647541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dbedc-e1fc-46e9-91db-2847d2e79645 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:08.727208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:08.764727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dbedc-e1fc-46e9-91db-2847d2e79645 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:08.799349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19283308-e959-49f6-ac4b-869167848ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:08.799997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283308-e959-49f6-ac4b-869167848ee4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:08.800280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283308-e959-49f6-ac4b-869167848ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:08.800555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283308-e959-49f6-ac4b-869167848ee4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:08.801381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19283308-e959-49f6-ac4b-869167848ee4 None admin] RBAC: Adding query filter params (role.id=0f21ddda19344dfd855c441dd59142ef, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=ad06f31859f9413f9db6039ac2abf982) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:08.801776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19283308-e959-49f6-ac4b-869167848ee4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:08.803155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19283308-e959-49f6-ac4b-869167848ee4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:08.817671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 522/1045] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:41:08 2022] GET /identity/v3/role_assignments?user.id=ad06f31859f9413f9db6039ac2abf982&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0f21ddda19344dfd855c441dd59142ef => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:09.469273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaca63b-7101-4360-896c-7e0dfd94fdcb None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:09.469659 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaca63b-7101-4360-896c-7e0dfd94fdcb None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:09.469870 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaca63b-7101-4360-896c-7e0dfd94fdcb None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:09.471290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 524/1046] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:09.476334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0df17f-3bac-4b4a-8ea5-28e76c097392 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:09.476758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0df17f-3bac-4b4a-8ea5-28e76c097392 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:09.476916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0df17f-3bac-4b4a-8ea5-28e76c097392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:09.493752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-fa0df17f-3bac-4b4a-8ea5-28e76c097392 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:09.494938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-fa0df17f-3bac-4b4a-8ea5-28e76c097392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:09.572477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 523/1047] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:09.596987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaca63b-7101-4360-896c-7e0dfd94fdcb None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:09.665977 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:09.703482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaca63b-7101-4360-896c-7e0dfd94fdcb None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:09.737492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0d7b7-7002-4f25-8d95-b738b0cf09f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:09.738237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d7b7-7002-4f25-8d95-b738b0cf09f7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:09.738517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d7b7-7002-4f25-8d95-b738b0cf09f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:09.738788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d7b7-7002-4f25-8d95-b738b0cf09f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:09.748875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0d7b7-7002-4f25-8d95-b738b0cf09f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-38f0d7b7-7002-4f25-8d95-b738b0cf09f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:09.756689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:09.757865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:09.758726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:09.761722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 525/1048] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:41:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:09.764599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0df17f-3bac-4b4a-8ea5-28e76c097392 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:09.822646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:09.866255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0df17f-3bac-4b4a-8ea5-28e76c097392 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:09.900637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4497edd-ffbe-45e6-aef0-5d3b421c099f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:09.901310 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497edd-ffbe-45e6-aef0-5d3b421c099f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:09.901615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497edd-ffbe-45e6-aef0-5d3b421c099f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:09.901888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497edd-ffbe-45e6-aef0-5d3b421c099f None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:09.902628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4497edd-ffbe-45e6-aef0-5d3b421c099f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:09.902927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4497edd-ffbe-45e6-aef0-5d3b421c099f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:09.904861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4497edd-ffbe-45e6-aef0-5d3b421c099f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:09.917626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 524/1049] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:41:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:09.919854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0d7b7-7002-4f25-8d95-b738b0cf09f7 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:10.024791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0d7b7-7002-4f25-8d95-b738b0cf09f7 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:10.050794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:10.061098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e1f87-2cbc-4d0c-90ce-61a0afebc159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:10.062056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e1f87-2cbc-4d0c-90ce-61a0afebc159 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.062337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e1f87-2cbc-4d0c-90ce-61a0afebc159 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.062608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e1f87-2cbc-4d0c-90ce-61a0afebc159 None admin] PATH_INFO: `/v3/users/4be7cf089c0f445e8eaf9c7693174d2a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.076733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e1f87-2cbc-4d0c-90ce-61a0afebc159 None admin] RBAC: Authorizing `identity:get_user(user_id=4be7cf089c0f445e8eaf9c7693174d2a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:10.078221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e1f87-2cbc-4d0c-90ce-61a0afebc159 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:10.080152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 526/1050] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:09 2022] GET /identity/v3/users/4be7cf089c0f445e8eaf9c7693174d2a => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:10.085924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4497edd-ffbe-45e6-aef0-5d3b421c099f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:10.195876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4497edd-ffbe-45e6-aef0-5d3b421c099f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:10.219827 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:10.232632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634166-7766-4366-9723-22459a8e152e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:10.233275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634166-7766-4366-9723-22459a8e152e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.233551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634166-7766-4366-9723-22459a8e152e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.233848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62634166-7766-4366-9723-22459a8e152e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.280439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc643e4d-53fd-4aa0-b12f-b7307278eb00 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.280680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc643e4d-53fd-4aa0-b12f-b7307278eb00 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.281080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc643e4d-53fd-4aa0-b12f-b7307278eb00 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.283069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 527/1051] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:41:10.290314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f22e0-4ea3-4758-954b-3d05d2a6468f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.290314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f22e0-4ea3-4758-954b-3d05d2a6468f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.290314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f22e0-4ea3-4758-954b-3d05d2a6468f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.305628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-0f2f22e0-4ea3-4758-954b-3d05d2a6468f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:10.306028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62634166-7766-4366-9723-22459a8e152e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:10.307381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0f2f22e0-4ea3-4758-954b-3d05d2a6468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:10.307494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62634166-7766-4366-9723-22459a8e152e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:10.378934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 528/1052] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:41:10.402832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f22e0-4ea3-4758-954b-3d05d2a6468f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:10.419409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 525/1053] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:10 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:41:10.421100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634166-7766-4366-9723-22459a8e152e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:10.444883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:10.511935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f22e0-4ea3-4758-954b-3d05d2a6468f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:10.529640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:10.536841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62634166-7766-4366-9723-22459a8e152e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:10.555947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2b6f8-fd16-46b8-97f4-2994d23a5fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:10.556736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2b6f8-fd16-46b8-97f4-2994d23a5fed None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.557013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2b6f8-fd16-46b8-97f4-2994d23a5fed None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.557284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2b6f8-fd16-46b8-97f4-2994d23a5fed None admin] PATH_INFO: `/v3/services/image` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.557992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2b6f8-fd16-46b8-97f4-2994d23a5fed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:10.559364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2b6f8-fd16-46b8-97f4-2994d23a5fed None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-0ca2b6f8-fd16-46b8-97f4-2994d23a5fed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:10.572050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:10.573130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 19:41:10.574025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:10.576718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dee010-23e4-4e74-bed8-6da08e7664d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:10.577685 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dee010-23e4-4e74-bed8-6da08e7664d5 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.578202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dee010-23e4-4e74-bed8-6da08e7664d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.578349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 529/1054] 10.222.0.66 () {56 vars in 1105 bytes} [Fri Jul 1 19:41:10 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:10.578442 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dee010-23e4-4e74-bed8-6da08e7664d5 None admin] PATH_INFO: `/v3/projects/d8164b89432449f08d1bc8aaa877452b` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.579181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2b6f8-fd16-46b8-97f4-2994d23a5fed None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:10.579737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dee010-23e4-4e74-bed8-6da08e7664d5 None admin] RBAC: Authorizing `identity:get_project(project_id=d8164b89432449f08d1bc8aaa877452b)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:10.581247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dee010-23e4-4e74-bed8-6da08e7664d5 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:10.583674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 526/1055] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:10 2022] GET /identity/v3/projects/d8164b89432449f08d1bc8aaa877452b => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:10.589237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dee010-23e4-4e74-bed8-6da08e7664d5 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:10.687001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:10.694103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2b6f8-fd16-46b8-97f4-2994d23a5fed None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:10.700283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dee010-23e4-4e74-bed8-6da08e7664d5 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:10.729636 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a4526-21db-4075-94ae-70a8ae331dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:10.730298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a4526-21db-4075-94ae-70a8ae331dab None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.730578 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a4526-21db-4075-94ae-70a8ae331dab None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.730857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a4526-21db-4075-94ae-70a8ae331dab None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.731553 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a4526-21db-4075-94ae-70a8ae331dab None admin] RBAC: Adding query filter params (name=image) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:10.732557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a4526-21db-4075-94ae-70a8ae331dab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:10.734009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a4526-21db-4075-94ae-70a8ae331dab None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:10.736408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0cc83f-b595-4aa5-992c-86dc9dda9fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:10.737062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cc83f-b595-4aa5-992c-86dc9dda9fec None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.737363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cc83f-b595-4aa5-992c-86dc9dda9fec None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.737635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0cc83f-b595-4aa5-992c-86dc9dda9fec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.738461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cc83f-b595-4aa5-992c-86dc9dda9fec None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=d8164b89432449f08d1bc8aaa877452b, user.id=4be7cf089c0f445e8eaf9c7693174d2a) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:10.738768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cc83f-b595-4aa5-992c-86dc9dda9fec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:10.740183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0cc83f-b595-4aa5-992c-86dc9dda9fec None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:10.744122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 530/1056] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:10.744874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a4526-21db-4075-94ae-70a8ae331dab None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:10.753845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 527/1057] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:41:10 2022] GET /identity/v3/role_assignments?user.id=4be7cf089c0f445e8eaf9c7693174d2a&scope.project.id=d8164b89432449f08d1bc8aaa877452b&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:10.801207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:10.867309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a4526-21db-4075-94ae-70a8ae331dab None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:10.897707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739c8f0-a871-41af-8d30-059a8419becf None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.898046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739c8f0-a871-41af-8d30-059a8419becf None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.898262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739c8f0-a871-41af-8d30-059a8419becf None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.902211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 528/1058] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:41:10.905768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913039e3-c584-45e8-86c1-e8239f879214 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.906046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913039e3-c584-45e8-86c1-e8239f879214 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.906146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa40dc8-4cb2-436e-9eda-c2d4ecdff00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:10.906314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913039e3-c584-45e8-86c1-e8239f879214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.906772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40dc8-4cb2-436e-9eda-c2d4ecdff00e None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:10.907054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40dc8-4cb2-436e-9eda-c2d4ecdff00e None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:10.907329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40dc8-4cb2-436e-9eda-c2d4ecdff00e None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:10.908043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40dc8-4cb2-436e-9eda-c2d4ecdff00e None admin] RBAC: Adding query filter params (type=image) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:10.908377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40dc8-4cb2-436e-9eda-c2d4ecdff00e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:10.910027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40dc8-4cb2-436e-9eda-c2d4ecdff00e None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:10.920300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 531/1059] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:10.921770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa40dc8-4cb2-436e-9eda-c2d4ecdff00e None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:10.927269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-913039e3-c584-45e8-86c1-e8239f879214 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:10.928455 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-913039e3-c584-45e8-86c1-e8239f879214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:10.999666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 529/1060] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:41:11.022343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913039e3-c584-45e8-86c1-e8239f879214 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:11.028166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa40dc8-4cb2-436e-9eda-c2d4ecdff00e None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:11.036360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:11.067944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f7149-0c1d-4383-b025-cf0b2fdce6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:11.068871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f7149-0c1d-4383-b025-cf0b2fdce6a0 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:11.069318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f7149-0c1d-4383-b025-cf0b2fdce6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:11.069882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f7149-0c1d-4383-b025-cf0b2fdce6a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:11.070896 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f7149-0c1d-4383-b025-cf0b2fdce6a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:11.072430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f7149-0c1d-4383-b025-cf0b2fdce6a0 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:11.094170 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 532/1061] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 1 19:41:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:41:11.117664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:11.136575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913039e3-c584-45e8-86c1-e8239f879214 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:11.185314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd74ef6-c8c1-48ec-ba3b-185b81640926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:11.186537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd74ef6-c8c1-48ec-ba3b-185b81640926 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:11.186822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd74ef6-c8c1-48ec-ba3b-185b81640926 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:11.187099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd74ef6-c8c1-48ec-ba3b-185b81640926 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:11.198884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd74ef6-c8c1-48ec-ba3b-185b81640926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-7cd74ef6-c8c1-48ec-ba3b-185b81640926 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:11.208426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:11.209465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:41:11.210373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:11.214002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 530/1062] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 1 19:41:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:11.215890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f7149-0c1d-4383-b025-cf0b2fdce6a0 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:11.322447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f7149-0c1d-4383-b025-cf0b2fdce6a0 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:11.355592 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92477644-1721-4485-8566-c4c87cabfa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:11.356306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92477644-1721-4485-8566-c4c87cabfa27 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:11.356592 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92477644-1721-4485-8566-c4c87cabfa27 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:11.357537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92477644-1721-4485-8566-c4c87cabfa27 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:11.358318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92477644-1721-4485-8566-c4c87cabfa27 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:11.358620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92477644-1721-4485-8566-c4c87cabfa27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:11.359984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92477644-1721-4485-8566-c4c87cabfa27 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:11.373523 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 533/1063] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 1 19:41:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:11.375248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd74ef6-c8c1-48ec-ba3b-185b81640926 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:11.481346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd74ef6-c8c1-48ec-ba3b-185b81640926 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:11.518736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588482-1a37-4bbd-abed-706b65308f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:11.519534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588482-1a37-4bbd-abed-706b65308f3c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:11.519816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588482-1a37-4bbd-abed-706b65308f3c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:11.520088 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc588482-1a37-4bbd-abed-706b65308f3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:11.529872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:11.532627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc588482-1a37-4bbd-abed-706b65308f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:11.534451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc588482-1a37-4bbd-abed-706b65308f3c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-cc588482-1a37-4bbd-abed-706b65308f3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:41:11.552755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:11.557872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:11.559157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:41:11.559157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:41:11.559157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:11.559157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:11.559157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 531/1064] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:11.562587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92477644-1721-4485-8566-c4c87cabfa27 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:11.590387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:11.672016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92477644-1721-4485-8566-c4c87cabfa27 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:11.706466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b2f95-cad1-4103-b1d7-01baffe5e848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:11.707124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b2f95-cad1-4103-b1d7-01baffe5e848 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:11.707396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b2f95-cad1-4103-b1d7-01baffe5e848 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:11.707666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b2f95-cad1-4103-b1d7-01baffe5e848 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:11.708378 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b2f95-cad1-4103-b1d7-01baffe5e848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:11.708680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b2f95-cad1-4103-b1d7-01baffe5e848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:11.710104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b2f95-cad1-4103-b1d7-01baffe5e848 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:11.722133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 534/1065] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:11.727982 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588482-1a37-4bbd-abed-706b65308f3c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:11.828148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc588482-1a37-4bbd-abed-706b65308f3c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:11.861321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec365d-efdc-4720-ae59-105b8002f2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:11.862057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec365d-efdc-4720-ae59-105b8002f2cb None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:11.862335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec365d-efdc-4720-ae59-105b8002f2cb None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:11.862608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec365d-efdc-4720-ae59-105b8002f2cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:11.870270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec365d-efdc-4720-ae59-105b8002f2cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:11.871711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec365d-efdc-4720-ae59-105b8002f2cb None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-86ec365d-efdc-4720-ae59-105b8002f2cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:41:11.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:11.888953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:11.890445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 532/1066] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 1 19:41:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:11.895031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b2f95-cad1-4103-b1d7-01baffe5e848 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:11.980670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:11.998367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b2f95-cad1-4103-b1d7-01baffe5e848 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:12.031405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8124f37-b809-416d-891f-24b0add21315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:12.032068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8124f37-b809-416d-891f-24b0add21315 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:12.032347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8124f37-b809-416d-891f-24b0add21315 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:12.032618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8124f37-b809-416d-891f-24b0add21315 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:12.033442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8124f37-b809-416d-891f-24b0add21315 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:12.033776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8124f37-b809-416d-891f-24b0add21315 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:12.035453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8124f37-b809-416d-891f-24b0add21315 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:12.057302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 535/1067] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 1 19:41:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:12.058986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec365d-efdc-4720-ae59-105b8002f2cb None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:12.113701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:12.163612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec365d-efdc-4720-ae59-105b8002f2cb None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:12.197441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cf3ab-8808-4d9e-8da2-620c4845091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:12.198270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf3ab-8808-4d9e-8da2-620c4845091d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:12.198552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf3ab-8808-4d9e-8da2-620c4845091d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:12.198826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf3ab-8808-4d9e-8da2-620c4845091d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:12.210163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cf3ab-8808-4d9e-8da2-620c4845091d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:12.211622 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cf3ab-8808-4d9e-8da2-620c4845091d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-0b8cf3ab-8808-4d9e-8da2-620c4845091d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:41:12.228623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:12.229539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:12.230474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:41:12.230474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:41:12.230474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:12.230474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:12.233664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 533/1068] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:12.236405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8124f37-b809-416d-891f-24b0add21315 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:12.350631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8124f37-b809-416d-891f-24b0add21315 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:12.385043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea85b94-5f43-4731-a4ea-77b5edfc5deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:12.385689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea85b94-5f43-4731-a4ea-77b5edfc5deb None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:12.385970 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea85b94-5f43-4731-a4ea-77b5edfc5deb None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:12.386242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea85b94-5f43-4731-a4ea-77b5edfc5deb None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:12.386951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea85b94-5f43-4731-a4ea-77b5edfc5deb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:12.387251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea85b94-5f43-4731-a4ea-77b5edfc5deb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:12.389166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea85b94-5f43-4731-a4ea-77b5edfc5deb None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:12.405006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 536/1069] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:12.406433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cf3ab-8808-4d9e-8da2-620c4845091d None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:12.511421 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cf3ab-8808-4d9e-8da2-620c4845091d None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:12.514323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:12.551333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764f639-2594-488d-bb28-0cdfb7c32075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:12.552126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764f639-2594-488d-bb28-0cdfb7c32075 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:12.552406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764f639-2594-488d-bb28-0cdfb7c32075 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:12.552676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764f639-2594-488d-bb28-0cdfb7c32075 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:12.559563 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764f639-2594-488d-bb28-0cdfb7c32075 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:12.561786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764f639-2594-488d-bb28-0cdfb7c32075 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-1764f639-2594-488d-bb28-0cdfb7c32075 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:41:12.576436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:12.577312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:12.578226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:41:12.578226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:41:12.578226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:41:12.578226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:41:12.578226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:41:12.578226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:41:12.578226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:41:12.578226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:12.578226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 534/1070] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:41:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:12.584716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea85b94-5f43-4731-a4ea-77b5edfc5deb None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:12.587614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:12.692334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea85b94-5f43-4731-a4ea-77b5edfc5deb None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:12.726351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce632d5d-f5b0-4205-b99c-a270fbcf2e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:12.727020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce632d5d-f5b0-4205-b99c-a270fbcf2e85 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:12.727266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce632d5d-f5b0-4205-b99c-a270fbcf2e85 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:12.727528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce632d5d-f5b0-4205-b99c-a270fbcf2e85 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:12.728308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce632d5d-f5b0-4205-b99c-a270fbcf2e85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:12.728608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce632d5d-f5b0-4205-b99c-a270fbcf2e85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:12.730051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce632d5d-f5b0-4205-b99c-a270fbcf2e85 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:12.748740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 537/1071] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:41:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:12.750034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764f639-2594-488d-bb28-0cdfb7c32075 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:12.835252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33f825-547d-4480-b482-eefbac579acc None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:12.835618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33f825-547d-4480-b482-eefbac579acc None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:12.835818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33f825-547d-4480-b482-eefbac579acc None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:12.837202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 538/1072] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:41:12.842534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53049be1-33a9-4ed3-9f22-2658b6466e79 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:12.842826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53049be1-33a9-4ed3-9f22-2658b6466e79 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:12.843104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53049be1-33a9-4ed3-9f22-2658b6466e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:12.855001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764f639-2594-488d-bb28-0cdfb7c32075 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:12.859824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-53049be1-33a9-4ed3-9f22-2658b6466e79 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:12.861014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-53049be1-33a9-4ed3-9f22-2658b6466e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:12.891696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc7d7c-1052-46dd-b4d6-dbc47fb030a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:12.892600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc7d7c-1052-46dd-b4d6-dbc47fb030a0 None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:12.892876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc7d7c-1052-46dd-b4d6-dbc47fb030a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:12.893239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc7d7c-1052-46dd-b4d6-dbc47fb030a0 None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/ad06f31859f9413f9db6039ac2abf982/roles/0f21ddda19344dfd855c441dd59142ef` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:12.895440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc7d7c-1052-46dd-b4d6-dbc47fb030a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=ad06f31859f9413f9db6039ac2abf982, role_id=0f21ddda19344dfd855c441dd59142ef)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:12.896864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc7d7c-1052-46dd-b4d6-dbc47fb030a0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:12.911666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 535/1073] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:41:12 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/ad06f31859f9413f9db6039ac2abf982/roles/0f21ddda19344dfd855c441dd59142ef => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:41:12.925516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:12.945736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 539/1074] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:12.965247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc7d7c-1052-46dd-b4d6-dbc47fb030a0 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:12.995707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:13.085972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc7d7c-1052-46dd-b4d6-dbc47fb030a0 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:13.122093 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5598ddb-a90a-4630-87cb-0c3a4f722763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:13.123749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5598ddb-a90a-4630-87cb-0c3a4f722763 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:13.124230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5598ddb-a90a-4630-87cb-0c3a4f722763 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:13.124684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5598ddb-a90a-4630-87cb-0c3a4f722763 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:13.135374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5598ddb-a90a-4630-87cb-0c3a4f722763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-d5598ddb-a90a-4630-87cb-0c3a4f722763 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:13.143937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:13.144919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:13.146462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 536/1075] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:41:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:13.151356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53049be1-33a9-4ed3-9f22-2658b6466e79 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:13.246925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9e970-e42f-4a8d-8cbb-fc8e22c88ae8 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:13.247928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9e970-e42f-4a8d-8cbb-fc8e22c88ae8 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:13.247928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9e970-e42f-4a8d-8cbb-fc8e22c88ae8 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:13.249659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 537/1076] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:41:13.254489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70512197-ea47-457a-93c1-0f74c135874b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:13.254688 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53049be1-33a9-4ed3-9f22-2658b6466e79 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:13.254873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70512197-ea47-457a-93c1-0f74c135874b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:13.255152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70512197-ea47-457a-93c1-0f74c135874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:13.273230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-70512197-ea47-457a-93c1-0f74c135874b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:13.274437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-70512197-ea47-457a-93c1-0f74c135874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:13.290087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e14fba-4e9d-4195-a69d-6bade8d429ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:13.290758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e14fba-4e9d-4195-a69d-6bade8d429ed None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:13.291037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e14fba-4e9d-4195-a69d-6bade8d429ed None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:13.291310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e14fba-4e9d-4195-a69d-6bade8d429ed None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:13.292054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e14fba-4e9d-4195-a69d-6bade8d429ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:13.292357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e14fba-4e9d-4195-a69d-6bade8d429ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:13.293756 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e14fba-4e9d-4195-a69d-6bade8d429ed None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:13.301910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 540/1077] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:41:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:13.307648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e14fba-4e9d-4195-a69d-6bade8d429ed None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:13.347860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 538/1078] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:41:13.365916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:13.372801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70512197-ea47-457a-93c1-0f74c135874b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:13.396033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:13.417696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e14fba-4e9d-4195-a69d-6bade8d429ed None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:13.453230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c081815-baa4-4a92-b818-681daf312a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:13.454217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c081815-baa4-4a92-b818-681daf312a71 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:13.454496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c081815-baa4-4a92-b818-681daf312a71 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:13.454770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c081815-baa4-4a92-b818-681daf312a71 None admin] PATH_INFO: `/v3/users/4be7cf089c0f445e8eaf9c7693174d2a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:13.455957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c081815-baa4-4a92-b818-681daf312a71 None admin] RBAC: Authorizing `identity:get_user(user_id=4be7cf089c0f445e8eaf9c7693174d2a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:13.457386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c081815-baa4-4a92-b818-681daf312a71 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:13.463855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 541/1079] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:13 2022] GET /identity/v3/users/4be7cf089c0f445e8eaf9c7693174d2a => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:13.465687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c081815-baa4-4a92-b818-681daf312a71 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:13.495025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70512197-ea47-457a-93c1-0f74c135874b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:13.531404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc9215-95eb-496b-8d09-e8176096f526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:13.531934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc9215-95eb-496b-8d09-e8176096f526 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:13.532215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc9215-95eb-496b-8d09-e8176096f526 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:13.532488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc9215-95eb-496b-8d09-e8176096f526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:13.583159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c081815-baa4-4a92-b818-681daf312a71 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:13.612660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:13.618666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc9215-95eb-496b-8d09-e8176096f526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:13.620127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc9215-95eb-496b-8d09-e8176096f526 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:13.631953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192d2df-afb9-40b1-badb-37b59b8f24d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:13.632805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192d2df-afb9-40b1-badb-37b59b8f24d7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:13.633105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192d2df-afb9-40b1-badb-37b59b8f24d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:13.633347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192d2df-afb9-40b1-badb-37b59b8f24d7 None admin] PATH_INFO: `/v3/projects/d8164b89432449f08d1bc8aaa877452b` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:13.634553 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192d2df-afb9-40b1-badb-37b59b8f24d7 None admin] RBAC: Authorizing `identity:get_project(project_id=d8164b89432449f08d1bc8aaa877452b)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:13.635969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192d2df-afb9-40b1-badb-37b59b8f24d7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:13.642923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 542/1080] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:13 2022] GET /identity/v3/projects/d8164b89432449f08d1bc8aaa877452b => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:13.643108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:13.644619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192d2df-afb9-40b1-badb-37b59b8f24d7 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:13.736985 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 539/1081] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:41:13.743347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc9215-95eb-496b-8d09-e8176096f526 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:13.756599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192d2df-afb9-40b1-badb-37b59b8f24d7 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:13.801721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6655b-d52e-4bdf-8530-4901acfdba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:13.802524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6655b-d52e-4bdf-8530-4901acfdba42 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:13.802837 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6655b-d52e-4bdf-8530-4901acfdba42 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:13.803117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6655b-d52e-4bdf-8530-4901acfdba42 None admin] PATH_INFO: `/v3/projects/d8164b89432449f08d1bc8aaa877452b/users/4be7cf089c0f445e8eaf9c7693174d2a/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:13.805108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6655b-d52e-4bdf-8530-4901acfdba42 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8164b89432449f08d1bc8aaa877452b, user_id=4be7cf089c0f445e8eaf9c7693174d2a, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:13.806573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6655b-d52e-4bdf-8530-4901acfdba42 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:13.822622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 543/1082] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:41:13 2022] PUT /identity/v3/projects/d8164b89432449f08d1bc8aaa877452b/users/4be7cf089c0f445e8eaf9c7693174d2a/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:41:13.864019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc9215-95eb-496b-8d09-e8176096f526 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:13.880734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:13.909046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d268bc-bbdc-49a7-ba5f-baa1fc7d9232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:13.910141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d268bc-bbdc-49a7-ba5f-baa1fc7d9232 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:13.910421 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d268bc-bbdc-49a7-ba5f-baa1fc7d9232 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:13.910689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d268bc-bbdc-49a7-ba5f-baa1fc7d9232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:13.922759 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d268bc-bbdc-49a7-ba5f-baa1fc7d9232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:13.924297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d268bc-bbdc-49a7-ba5f-baa1fc7d9232 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-24d268bc-bbdc-49a7-ba5f-baa1fc7d9232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:41:13.944291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:13.945243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:13.946173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:41:13.946173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:41:13.946173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:13.946173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:13.949972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 540/1083] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:13.952540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6655b-d52e-4bdf-8530-4901acfdba42 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:14.011274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:14.069103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6655b-d52e-4bdf-8530-4901acfdba42 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:14.105084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc642ed-e521-4308-9a84-6aad75a6f7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:14.105775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc642ed-e521-4308-9a84-6aad75a6f7dc None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:14.106059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc642ed-e521-4308-9a84-6aad75a6f7dc None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:14.106332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc642ed-e521-4308-9a84-6aad75a6f7dc None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:14.107213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc642ed-e521-4308-9a84-6aad75a6f7dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:14.107517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc642ed-e521-4308-9a84-6aad75a6f7dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:14.108894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc642ed-e521-4308-9a84-6aad75a6f7dc None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:14.125672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 544/1084] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:14.966517 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3568c96-5440-4581-8516-299977ad1372 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:14.967043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3568c96-5440-4581-8516-299977ad1372 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:14.967338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3568c96-5440-4581-8516-299977ad1372 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:14.968726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 541/1085] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:14.973830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28fa6e-e852-4c06-9e58-477bb92c1d36 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:14.974217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28fa6e-e852-4c06-9e58-477bb92c1d36 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:14.974376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28fa6e-e852-4c06-9e58-477bb92c1d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:14.990401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-0e28fa6e-e852-4c06-9e58-477bb92c1d36 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:14.991570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0e28fa6e-e852-4c06-9e58-477bb92c1d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:15.056181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 545/1086] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:15.080921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3568c96-5440-4581-8516-299977ad1372 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:15.179013 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3568c96-5440-4581-8516-299977ad1372 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:15.212111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbd6f1-e226-45db-add6-91fe735d4f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:15.212901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbd6f1-e226-45db-add6-91fe735d4f0e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:15.213177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbd6f1-e226-45db-add6-91fe735d4f0e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:15.213446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbd6f1-e226-45db-add6-91fe735d4f0e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:15.223831 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dbd6f1-e226-45db-add6-91fe735d4f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-e6dbd6f1-e226-45db-add6-91fe735d4f0e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:15.231968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:15.232974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 19:41:15.233945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:15.237738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 542/1087] 10.222.0.66 () {56 vars in 1120 bytes} [Fri Jul 1 19:41:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:15.239056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28fa6e-e852-4c06-9e58-477bb92c1d36 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:15.281456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:15.339020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28fa6e-e852-4c06-9e58-477bb92c1d36 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:15.372845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258386b8-4dd4-44ab-9aeb-4c86ce3d669a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:15.373573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258386b8-4dd4-44ab-9aeb-4c86ce3d669a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:15.373883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258386b8-4dd4-44ab-9aeb-4c86ce3d669a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:15.374153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258386b8-4dd4-44ab-9aeb-4c86ce3d669a None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:15.374888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258386b8-4dd4-44ab-9aeb-4c86ce3d669a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:15.375188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258386b8-4dd4-44ab-9aeb-4c86ce3d669a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:15.376549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258386b8-4dd4-44ab-9aeb-4c86ce3d669a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:15.384323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 546/1088] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 1 19:41:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:15.390762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbd6f1-e226-45db-add6-91fe735d4f0e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:15.418194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:15.487968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbd6f1-e226-45db-add6-91fe735d4f0e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:15.521561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000ed81-d88e-44e9-abf2-dd8926c1d4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:15.522369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000ed81-d88e-44e9-abf2-dd8926c1d4e0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:15.522645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000ed81-d88e-44e9-abf2-dd8926c1d4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:15.522910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000ed81-d88e-44e9-abf2-dd8926c1d4e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:15.533151 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000ed81-d88e-44e9-abf2-dd8926c1d4e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:15.534870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000ed81-d88e-44e9-abf2-dd8926c1d4e0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-7000ed81-d88e-44e9-abf2-dd8926c1d4e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:41:15.551446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:15.552498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:15.553461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:41:15.553461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:41:15.553461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:15.553461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:15.553461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 543/1089] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:15.558414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258386b8-4dd4-44ab-9aeb-4c86ce3d669a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:15.689248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258386b8-4dd4-44ab-9aeb-4c86ce3d669a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:15.690793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:15.724868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d177b7-13cd-4eb8-a5d7-b20c89385cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:15.725514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d177b7-13cd-4eb8-a5d7-b20c89385cc4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:15.725819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d177b7-13cd-4eb8-a5d7-b20c89385cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:15.726089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d177b7-13cd-4eb8-a5d7-b20c89385cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:15.726797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d177b7-13cd-4eb8-a5d7-b20c89385cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:15.727095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d177b7-13cd-4eb8-a5d7-b20c89385cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:15.728452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d177b7-13cd-4eb8-a5d7-b20c89385cc4 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:15.740261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 547/1090] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:15.745731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000ed81-d88e-44e9-abf2-dd8926c1d4e0 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:15.823591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:15.846079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000ed81-d88e-44e9-abf2-dd8926c1d4e0 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:15.876302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90ec7b-3d3f-42a0-84ba-788509f39af3 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:15.876595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90ec7b-3d3f-42a0-84ba-788509f39af3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:15.876875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90ec7b-3d3f-42a0-84ba-788509f39af3 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:15.878302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 548/1091] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:41:15.879450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ffbe7-e332-4926-8432-feb2d950c662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:15.880238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ffbe7-e332-4926-8432-feb2d950c662 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:15.880519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ffbe7-e332-4926-8432-feb2d950c662 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:15.880794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ffbe7-e332-4926-8432-feb2d950c662 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:15.883171 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccb555-f5e4-453b-9c8a-e3367f18332a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:15.883463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccb555-f5e4-453b-9c8a-e3367f18332a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:15.883744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccb555-f5e4-453b-9c8a-e3367f18332a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:15.888697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ffbe7-e332-4926-8432-feb2d950c662 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:15.890188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ffbe7-e332-4926-8432-feb2d950c662 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:15.901286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a9ccb555-f5e4-453b-9c8a-e3367f18332a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:15.902538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a9ccb555-f5e4-453b-9c8a-e3367f18332a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6a7ffbe7-e332-4926-8432-feb2d950c662 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:41:15.905143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:15.906130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:15.907082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 544/1092] 10.222.0.66 () {56 vars in 1093 bytes} [Fri Jul 1 19:41:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:15.912299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ffbe7-e332-4926-8432-feb2d950c662 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:15.972395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 549/1093] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:41:15.997129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccb555-f5e4-453b-9c8a-e3367f18332a None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.018976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ffbe7-e332-4926-8432-feb2d950c662 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:16.057776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a802ade-23f0-484c-8809-b0d8d6ec8f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:16.058438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a802ade-23f0-484c-8809-b0d8d6ec8f3b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:16.058713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a802ade-23f0-484c-8809-b0d8d6ec8f3b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:16.058988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a802ade-23f0-484c-8809-b0d8d6ec8f3b None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:16.060390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a802ade-23f0-484c-8809-b0d8d6ec8f3b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:16.060686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a802ade-23f0-484c-8809-b0d8d6ec8f3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:16.062141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a802ade-23f0-484c-8809-b0d8d6ec8f3b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:16.071875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:16.080309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 545/1094] 10.222.0.66 () {56 vars in 1133 bytes} [Fri Jul 1 19:41:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:16.086080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a802ade-23f0-484c-8809-b0d8d6ec8f3b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.091832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:16.100977 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccb555-f5e4-453b-9c8a-e3367f18332a None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:16.134866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5571f-db31-4f7f-a6ad-3802a50fc7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:16.135645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5571f-db31-4f7f-a6ad-3802a50fc7e4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:16.135922 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5571f-db31-4f7f-a6ad-3802a50fc7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:16.136192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5571f-db31-4f7f-a6ad-3802a50fc7e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:16.147366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5571f-db31-4f7f-a6ad-3802a50fc7e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-38d5571f-db31-4f7f-a6ad-3802a50fc7e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:16.156186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:16.157239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:16.158110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 550/1095] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:41:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:16.163793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5571f-db31-4f7f-a6ad-3802a50fc7e4 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.185850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a802ade-23f0-484c-8809-b0d8d6ec8f3b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:16.219389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89ca7f-c3a6-44ab-a739-7a796d75bc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:16.220027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89ca7f-c3a6-44ab-a739-7a796d75bc3b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:16.220306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89ca7f-c3a6-44ab-a739-7a796d75bc3b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:16.220584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89ca7f-c3a6-44ab-a739-7a796d75bc3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:16.264501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5571f-db31-4f7f-a6ad-3802a50fc7e4 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:16.280872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:16.287832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89ca7f-c3a6-44ab-a739-7a796d75bc3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:16.289271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89ca7f-c3a6-44ab-a739-7a796d75bc3b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:16.300985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fb008-fa36-4d95-9381-aec6cd58f6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:16.301669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fb008-fa36-4d95-9381-aec6cd58f6e1 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:16.301948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fb008-fa36-4d95-9381-aec6cd58f6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:16.302218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fb008-fa36-4d95-9381-aec6cd58f6e1 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:16.302951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fb008-fa36-4d95-9381-aec6cd58f6e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:16.303250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fb008-fa36-4d95-9381-aec6cd58f6e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:16.304614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fb008-fa36-4d95-9381-aec6cd58f6e1 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:16.312967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 551/1096] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:16.318723 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fb008-fa36-4d95-9381-aec6cd58f6e1 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.334769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:16.393042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 546/1097] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:41:16.398721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89ca7f-c3a6-44ab-a739-7a796d75bc3b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.425994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fb008-fa36-4d95-9381-aec6cd58f6e1 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:16.459971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ea723-3122-4e7b-8054-745ccc96ea3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:16.460836 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ea723-3122-4e7b-8054-745ccc96ea3c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:16.461114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ea723-3122-4e7b-8054-745ccc96ea3c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:16.461387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ea723-3122-4e7b-8054-745ccc96ea3c None admin] PATH_INFO: `/v3/users/4be7cf089c0f445e8eaf9c7693174d2a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:16.462602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ea723-3122-4e7b-8054-745ccc96ea3c None admin] RBAC: Authorizing `identity:get_user(user_id=4be7cf089c0f445e8eaf9c7693174d2a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:16.464061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ea723-3122-4e7b-8054-745ccc96ea3c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:16.465722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 552/1098] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:16 2022] GET /identity/v3/users/4be7cf089c0f445e8eaf9c7693174d2a => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:16.474499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ea723-3122-4e7b-8054-745ccc96ea3c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.495666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:16.503507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89ca7f-c3a6-44ab-a739-7a796d75bc3b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:16.539899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657bd97-284d-4eac-beb2-0a9532293179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:16.540603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657bd97-284d-4eac-beb2-0a9532293179 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:16.540877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657bd97-284d-4eac-beb2-0a9532293179 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:16.541145 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657bd97-284d-4eac-beb2-0a9532293179 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:16.552215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657bd97-284d-4eac-beb2-0a9532293179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:16.553683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657bd97-284d-4eac-beb2-0a9532293179 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:16.558033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-f657bd97-284d-4eac-beb2-0a9532293179 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:41:16.572216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:16.573142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:16.574072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:41:16.574072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:41:16.574072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:41:16.574072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:16.574072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 547/1099] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:16.578030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ea723-3122-4e7b-8054-745ccc96ea3c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:16.579536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657bd97-284d-4eac-beb2-0a9532293179 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.612711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6c1d5-f7a9-4a41-a3e0-b8270abdfa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:16.613385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6c1d5-f7a9-4a41-a3e0-b8270abdfa4a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:16.613716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6c1d5-f7a9-4a41-a3e0-b8270abdfa4a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:16.613995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6c1d5-f7a9-4a41-a3e0-b8270abdfa4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:16.680580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6c1d5-f7a9-4a41-a3e0-b8270abdfa4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:16.682073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6c1d5-f7a9-4a41-a3e0-b8270abdfa4a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:16.688470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657bd97-284d-4eac-beb2-0a9532293179 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:16.724958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe45a3-2fcd-4958-ab21-6d2f1da3d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:16.725647 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe45a3-2fcd-4958-ab21-6d2f1da3d1a0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:16.725942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe45a3-2fcd-4958-ab21-6d2f1da3d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:16.726215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe45a3-2fcd-4958-ab21-6d2f1da3d1a0 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:16.726927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe45a3-2fcd-4958-ab21-6d2f1da3d1a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:16.727229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe45a3-2fcd-4958-ab21-6d2f1da3d1a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:16.728598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe45a3-2fcd-4958-ab21-6d2f1da3d1a0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:16.740064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 548/1100] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:16.745501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe45a3-2fcd-4958-ab21-6d2f1da3d1a0 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.756073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:16.767354 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:16.788452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 553/1101] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 19:41:16.794948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6c1d5-f7a9-4a41-a3e0-b8270abdfa4a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.846196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe45a3-2fcd-4958-ab21-6d2f1da3d1a0 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:16.878783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad0222-7667-4ca7-960f-b877b7de6c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:16.879575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad0222-7667-4ca7-960f-b877b7de6c95 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:16.879859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad0222-7667-4ca7-960f-b877b7de6c95 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:16.880136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad0222-7667-4ca7-960f-b877b7de6c95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:16.886695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad0222-7667-4ca7-960f-b877b7de6c95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:16.888782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad0222-7667-4ca7-960f-b877b7de6c95 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:16.891398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6c1d5-f7a9-4a41-a3e0-b8270abdfa4a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-b7ad0222-7667-4ca7-960f-b877b7de6c95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:41:16.902995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:16.904067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:16.905157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:41:16.905157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:41:16.905157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:41:16.905157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:41:16.905157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:41:16.905157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:41:16.905157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:41:16.905157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:16.905157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 549/1102] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:41:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:41:16.910367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad0222-7667-4ca7-960f-b877b7de6c95 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.925533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7ada7-363b-4f99-ae55-e3d4de0f6f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:16.926410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7ada7-363b-4f99-ae55-e3d4de0f6f97 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:16.926694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7ada7-363b-4f99-ae55-e3d4de0f6f97 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:16.926966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7ada7-363b-4f99-ae55-e3d4de0f6f97 None admin] PATH_INFO: `/v3/projects/d8164b89432449f08d1bc8aaa877452b` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:16.928128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7ada7-363b-4f99-ae55-e3d4de0f6f97 None admin] RBAC: Authorizing `identity:get_project(project_id=d8164b89432449f08d1bc8aaa877452b)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:16.929553 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7ada7-363b-4f99-ae55-e3d4de0f6f97 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:16.931275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 554/1103] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:16 2022] GET /identity/v3/projects/d8164b89432449f08d1bc8aaa877452b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:16.937656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7ada7-363b-4f99-ae55-e3d4de0f6f97 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:16.981197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:16.987040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:17.014583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad0222-7667-4ca7-960f-b877b7de6c95 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:17.037437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7ada7-363b-4f99-ae55-e3d4de0f6f97 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:17.048772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e2b86-df14-4290-97ed-0a97b8a2e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:17.049364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e2b86-df14-4290-97ed-0a97b8a2e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:17.049679 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e2b86-df14-4290-97ed-0a97b8a2e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:17.049953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e2b86-df14-4290-97ed-0a97b8a2e9e4 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:17.050734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e2b86-df14-4290-97ed-0a97b8a2e9e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:17.051044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e2b86-df14-4290-97ed-0a97b8a2e9e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:17.052459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e2b86-df14-4290-97ed-0a97b8a2e9e4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:17.064828 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 550/1104] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:41:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:17.070672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e2b86-df14-4290-97ed-0a97b8a2e9e4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:17.071099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2638da7-e4bb-4a04-ba7c-3ecde7644be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:17.071751 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2638da7-e4bb-4a04-ba7c-3ecde7644be7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:17.072031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2638da7-e4bb-4a04-ba7c-3ecde7644be7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:17.072307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2638da7-e4bb-4a04-ba7c-3ecde7644be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:17.073132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2638da7-e4bb-4a04-ba7c-3ecde7644be7 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=d8164b89432449f08d1bc8aaa877452b, user.id=4be7cf089c0f445e8eaf9c7693174d2a) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:17.073443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2638da7-e4bb-4a04-ba7c-3ecde7644be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:17.074912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2638da7-e4bb-4a04-ba7c-3ecde7644be7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:17.081386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 555/1105] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:41:16 2022] GET /identity/v3/role_assignments?user.id=4be7cf089c0f445e8eaf9c7693174d2a&scope.project.id=d8164b89432449f08d1bc8aaa877452b&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:17.175825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e2b86-df14-4290-97ed-0a97b8a2e9e4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:17.203845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:17.212053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a6c54-c4d2-4c39-b4f7-da4f33f34c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:17.212714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a6c54-c4d2-4c39-b4f7-da4f33f34c7d None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:17.212995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a6c54-c4d2-4c39-b4f7-da4f33f34c7d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:17.213263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a6c54-c4d2-4c39-b4f7-da4f33f34c7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:17.214136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a6c54-c4d2-4c39-b4f7-da4f33f34c7d None admin] RBAC: Adding query filter params (role.id=0f21ddda19344dfd855c441dd59142ef, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=ad06f31859f9413f9db6039ac2abf982) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:17.214444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a6c54-c4d2-4c39-b4f7-da4f33f34c7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:17.216040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a6c54-c4d2-4c39-b4f7-da4f33f34c7d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:17.222911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 551/1106] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:41:17 2022] GET /identity/v3/role_assignments?user.id=ad06f31859f9413f9db6039ac2abf982&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0f21ddda19344dfd855c441dd59142ef => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:19.203979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1cbd3-475a-464a-9d57-861c652206ec None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:19.204564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1cbd3-475a-464a-9d57-861c652206ec None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:19.204564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1cbd3-475a-464a-9d57-861c652206ec None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:19.205931 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 556/1107] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:19.210992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890df05c-ea33-4564-b103-b8c049c71aa4 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:19.211407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890df05c-ea33-4564-b103-b8c049c71aa4 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:19.211544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890df05c-ea33-4564-b103-b8c049c71aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:19.228307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-890df05c-ea33-4564-b103-b8c049c71aa4 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:19.229476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-890df05c-ea33-4564-b103-b8c049c71aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:19.295706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 552/1108] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:19.320167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1cbd3-475a-464a-9d57-861c652206ec None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:19.420172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1cbd3-475a-464a-9d57-861c652206ec None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:19.433574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:19.454485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11b2e0-a772-48f3-ba0f-30edc452b2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:19.455196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11b2e0-a772-48f3-ba0f-30edc452b2f6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:19.455472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11b2e0-a772-48f3-ba0f-30edc452b2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:19.455741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11b2e0-a772-48f3-ba0f-30edc452b2f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:19.456889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11b2e0-a772-48f3-ba0f-30edc452b2f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:19.458348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11b2e0-a772-48f3-ba0f-30edc452b2f6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:19.459951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 557/1109] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:19.465496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890df05c-ea33-4564-b103-b8c049c71aa4 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:19.563148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890df05c-ea33-4564-b103-b8c049c71aa4 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:19.585213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:19.597427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9036f5-eb98-447d-ba55-8b8251cf0b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:19.598133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9036f5-eb98-447d-ba55-8b8251cf0b1d None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:19.598414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9036f5-eb98-447d-ba55-8b8251cf0b1d None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:19.598687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9036f5-eb98-447d-ba55-8b8251cf0b1d None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:19.599390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9036f5-eb98-447d-ba55-8b8251cf0b1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:19.600815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9036f5-eb98-447d-ba55-8b8251cf0b1d None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:19.619425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 553/1110] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:41:19 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:41:21.718991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511de36-dd2f-483a-a31c-4e71a7d339fe None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:21.719566 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511de36-dd2f-483a-a31c-4e71a7d339fe None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:21.719566 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511de36-dd2f-483a-a31c-4e71a7d339fe None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:21.720904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 558/1111] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:21.726143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9f436-4803-4284-a026-ae2194b8a167 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:21.726569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9f436-4803-4284-a026-ae2194b8a167 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:21.726695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9f436-4803-4284-a026-ae2194b8a167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:21.743330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a5d9f436-4803-4284-a026-ae2194b8a167 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:21.744500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a5d9f436-4803-4284-a026-ae2194b8a167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:21.810335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 554/1112] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:21.834234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511de36-dd2f-483a-a31c-4e71a7d339fe None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:21.933396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511de36-dd2f-483a-a31c-4e71a7d339fe None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:21.967060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe12ba1-04fa-424b-adda-70028abef01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:21.967905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe12ba1-04fa-424b-adda-70028abef01c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:21.968194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe12ba1-04fa-424b-adda-70028abef01c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:21.968469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe12ba1-04fa-424b-adda-70028abef01c None admin] PATH_INFO: `/v3/roles/b127031d393048d7ae9a35015113e82a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:21.969976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe12ba1-04fa-424b-adda-70028abef01c None admin] RBAC: Authorizing `identity:get_role(role_id=b127031d393048d7ae9a35015113e82a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:21.971354 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe12ba1-04fa-424b-adda-70028abef01c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:21.972720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 559/1113] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:21 2022] GET /identity/v3/roles/b127031d393048d7ae9a35015113e82a => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:21.978459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9f436-4803-4284-a026-ae2194b8a167 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:22.074508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9f436-4803-4284-a026-ae2194b8a167 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:22.083338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:22.109131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1b6e6-06b3-4867-81b4-e97f9fdf1773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:22.110020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1b6e6-06b3-4867-81b4-e97f9fdf1773 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:22.110301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1b6e6-06b3-4867-81b4-e97f9fdf1773 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:22.110574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1b6e6-06b3-4867-81b4-e97f9fdf1773 None admin] PATH_INFO: `/v3/users/5148eeb539b049b8a03158acc861dbce` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:22.123440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1b6e6-06b3-4867-81b4-e97f9fdf1773 None admin] RBAC: Authorizing `identity:get_user(user_id=5148eeb539b049b8a03158acc861dbce)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:22.124900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1b6e6-06b3-4867-81b4-e97f9fdf1773 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:22.126545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 555/1114] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:21 2022] GET /identity/v3/users/5148eeb539b049b8a03158acc861dbce => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:22.132345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe12ba1-04fa-424b-adda-70028abef01c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:22.181348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:22.229266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe12ba1-04fa-424b-adda-70028abef01c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:22.262090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51a39-7216-49ac-860c-43d572e5dbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:22.262979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51a39-7216-49ac-860c-43d572e5dbbb None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:22.263281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51a39-7216-49ac-860c-43d572e5dbbb None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:22.263559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d51a39-7216-49ac-860c-43d572e5dbbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:22.328878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d51a39-7216-49ac-860c-43d572e5dbbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:22.330275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d51a39-7216-49ac-860c-43d572e5dbbb None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:22.376372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:22.426189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 560/1115] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:41:22.432583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1b6e6-06b3-4867-81b4-e97f9fdf1773 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:22.530195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1b6e6-06b3-4867-81b4-e97f9fdf1773 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:22.563084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2082107-db44-4960-8b7d-d40da3c86a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:22.563940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2082107-db44-4960-8b7d-d40da3c86a58 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:22.564216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2082107-db44-4960-8b7d-d40da3c86a58 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:22.564486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2082107-db44-4960-8b7d-d40da3c86a58 None admin] PATH_INFO: `/v3/projects/d8164b89432449f08d1bc8aaa877452b` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:22.565677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2082107-db44-4960-8b7d-d40da3c86a58 None admin] RBAC: Authorizing `identity:get_project(project_id=d8164b89432449f08d1bc8aaa877452b)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:22.567106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2082107-db44-4960-8b7d-d40da3c86a58 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:22.568787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 556/1116] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:22 2022] GET /identity/v3/projects/d8164b89432449f08d1bc8aaa877452b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:22.575280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51a39-7216-49ac-860c-43d572e5dbbb None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:22.670525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d51a39-7216-49ac-860c-43d572e5dbbb None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:22.703461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa46e3-6772-4a36-98b0-ce6bc46d9672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:22.704089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa46e3-6772-4a36-98b0-ce6bc46d9672 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:22.704367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa46e3-6772-4a36-98b0-ce6bc46d9672 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:22.704637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa46e3-6772-4a36-98b0-ce6bc46d9672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:22.705450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa46e3-6772-4a36-98b0-ce6bc46d9672 None admin] RBAC: Adding query filter params (role.id=b127031d393048d7ae9a35015113e82a, scope.project.id=d8164b89432449f08d1bc8aaa877452b, user.id=5148eeb539b049b8a03158acc861dbce) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:22.705790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa46e3-6772-4a36-98b0-ce6bc46d9672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:22.707196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa46e3-6772-4a36-98b0-ce6bc46d9672 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:22.712437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 561/1117] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:41:22 2022] GET /identity/v3/role_assignments?user.id=5148eeb539b049b8a03158acc861dbce&scope.project.id=d8164b89432449f08d1bc8aaa877452b&role.id=b127031d393048d7ae9a35015113e82a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:24.716401 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ceb492-f43a-4d11-8567-f17baf026682 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:24.716939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ceb492-f43a-4d11-8567-f17baf026682 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:24.717043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ceb492-f43a-4d11-8567-f17baf026682 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:24.718414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 557/1118] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:24.723564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ed5d-6229-489f-a246-8a828aaa99fa None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:24.724007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ed5d-6229-489f-a246-8a828aaa99fa None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:24.724106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ed5d-6229-489f-a246-8a828aaa99fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:24.740457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-6a79ed5d-6229-489f-a246-8a828aaa99fa None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:24.741666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6a79ed5d-6229-489f-a246-8a828aaa99fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:24.750678 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:24.807948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 562/1119] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:24.830879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ceb492-f43a-4d11-8567-f17baf026682 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:24.865389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:24.932078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ceb492-f43a-4d11-8567-f17baf026682 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:24.964924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b13c0-cb87-4a66-9e77-005cbd55d611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:24.966440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b13c0-cb87-4a66-9e77-005cbd55d611 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:24.966720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b13c0-cb87-4a66-9e77-005cbd55d611 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:24.966992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b13c0-cb87-4a66-9e77-005cbd55d611 None admin] PATH_INFO: `/v3/roles/b127031d393048d7ae9a35015113e82a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:24.968459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b13c0-cb87-4a66-9e77-005cbd55d611 None admin] RBAC: Authorizing `identity:get_role(role_id=b127031d393048d7ae9a35015113e82a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:24.969860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b13c0-cb87-4a66-9e77-005cbd55d611 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:24.971215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 558/1120] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:24 2022] GET /identity/v3/roles/b127031d393048d7ae9a35015113e82a => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:24.977137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ed5d-6229-489f-a246-8a828aaa99fa None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:25.071960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ed5d-6229-489f-a246-8a828aaa99fa None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:25.106446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693efcf-2bff-445b-be17-32cfd3c9049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:25.107310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693efcf-2bff-445b-be17-32cfd3c9049d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:25.107586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693efcf-2bff-445b-be17-32cfd3c9049d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:25.107884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693efcf-2bff-445b-be17-32cfd3c9049d None admin] PATH_INFO: `/v3/users/5148eeb539b049b8a03158acc861dbce` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:25.109062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693efcf-2bff-445b-be17-32cfd3c9049d None admin] RBAC: Authorizing `identity:get_user(user_id=5148eeb539b049b8a03158acc861dbce)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:25.110536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693efcf-2bff-445b-be17-32cfd3c9049d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:25.112174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 563/1121] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:24 2022] GET /identity/v3/users/5148eeb539b049b8a03158acc861dbce => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:25.118228 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b13c0-cb87-4a66-9e77-005cbd55d611 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:25.217705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b13c0-cb87-4a66-9e77-005cbd55d611 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:25.219170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:25.251885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f57a7-62df-4b4d-9a19-769da1bc712b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:25.252730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f57a7-62df-4b4d-9a19-769da1bc712b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:25.253005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f57a7-62df-4b4d-9a19-769da1bc712b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:25.253274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f57a7-62df-4b4d-9a19-769da1bc712b None admin] PATH_INFO: `/v3/projects/d8164b89432449f08d1bc8aaa877452b` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:25.254458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f57a7-62df-4b4d-9a19-769da1bc712b None admin] RBAC: Authorizing `identity:get_project(project_id=d8164b89432449f08d1bc8aaa877452b)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:25.255866 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f57a7-62df-4b4d-9a19-769da1bc712b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:25.257513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 559/1122] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:25 2022] GET /identity/v3/projects/d8164b89432449f08d1bc8aaa877452b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:25.264606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693efcf-2bff-445b-be17-32cfd3c9049d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:25.279257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:25.363261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693efcf-2bff-445b-be17-32cfd3c9049d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:25.396366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96918c48-827c-4a70-ac79-692a768e4150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:25.397326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96918c48-827c-4a70-ac79-692a768e4150 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:25.397635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96918c48-827c-4a70-ac79-692a768e4150 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:25.397916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96918c48-827c-4a70-ac79-692a768e4150 None admin] PATH_INFO: `/v3/projects/d8164b89432449f08d1bc8aaa877452b/users/5148eeb539b049b8a03158acc861dbce/roles/b127031d393048d7ae9a35015113e82a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:25.400098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96918c48-827c-4a70-ac79-692a768e4150 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8164b89432449f08d1bc8aaa877452b, user_id=5148eeb539b049b8a03158acc861dbce, role_id=b127031d393048d7ae9a35015113e82a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:25.401530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96918c48-827c-4a70-ac79-692a768e4150 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:25.414605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 564/1123] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:41:25 2022] PUT /identity/v3/projects/d8164b89432449f08d1bc8aaa877452b/users/5148eeb539b049b8a03158acc861dbce/roles/b127031d393048d7ae9a35015113e82a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:41:27.445535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35144-1f3c-4d26-9fa5-f3311a95dca9 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:27.446114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35144-1f3c-4d26-9fa5-f3311a95dca9 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:27.446114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee35144-1f3c-4d26-9fa5-f3311a95dca9 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:27.447484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 560/1124] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:27.452765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a8291-728f-4781-a263-70035b2deec6 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:27.453162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a8291-728f-4781-a263-70035b2deec6 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:27.453309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a8291-728f-4781-a263-70035b2deec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:27.470923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-6e0a8291-728f-4781-a263-70035b2deec6 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:27.472129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6e0a8291-728f-4781-a263-70035b2deec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:27.539291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 565/1125] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:27.563152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee35144-1f3c-4d26-9fa5-f3311a95dca9 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:27.661931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee35144-1f3c-4d26-9fa5-f3311a95dca9 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:27.694339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dc126-d72b-4d05-a64d-045bde970cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:27.695173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dc126-d72b-4d05-a64d-045bde970cda None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:27.695446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dc126-d72b-4d05-a64d-045bde970cda None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:27.695764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dc126-d72b-4d05-a64d-045bde970cda None admin] PATH_INFO: `/v3/roles/b127031d393048d7ae9a35015113e82a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:27.697155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64dc126-d72b-4d05-a64d-045bde970cda None admin] RBAC: Authorizing `identity:get_role(role_id=b127031d393048d7ae9a35015113e82a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:27.698551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64dc126-d72b-4d05-a64d-045bde970cda None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:27.700584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 561/1126] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:27 2022] GET /identity/v3/roles/b127031d393048d7ae9a35015113e82a => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:27.706235 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a8291-728f-4781-a263-70035b2deec6 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:27.708403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:27.813957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a8291-728f-4781-a263-70035b2deec6 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:27.848802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97001e-16f0-463e-b474-76aef38377e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:27.849695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97001e-16f0-463e-b474-76aef38377e4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:27.849976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97001e-16f0-463e-b474-76aef38377e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:27.850246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97001e-16f0-463e-b474-76aef38377e4 None admin] PATH_INFO: `/v3/users/5148eeb539b049b8a03158acc861dbce` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:27.851416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97001e-16f0-463e-b474-76aef38377e4 None admin] RBAC: Authorizing `identity:get_user(user_id=5148eeb539b049b8a03158acc861dbce)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:27.852847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97001e-16f0-463e-b474-76aef38377e4 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:27.854512 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 566/1127] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:27 2022] GET /identity/v3/users/5148eeb539b049b8a03158acc861dbce => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:27.860324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dc126-d72b-4d05-a64d-045bde970cda None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:27.899127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:27.959041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dc126-d72b-4d05-a64d-045bde970cda None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:27.991685 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3754b-d4c2-48f5-b8f9-b45164d229fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:27.992318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3754b-d4c2-48f5-b8f9-b45164d229fb None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:27.992600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3754b-d4c2-48f5-b8f9-b45164d229fb None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:27.992875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3754b-d4c2-48f5-b8f9-b45164d229fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:28.058167 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3754b-d4c2-48f5-b8f9-b45164d229fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:28.059632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3754b-d4c2-48f5-b8f9-b45164d229fb None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:28.096745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:28.155241 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 562/1128] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:41:28.161407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97001e-16f0-463e-b474-76aef38377e4 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:28.228596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:28.258864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97001e-16f0-463e-b474-76aef38377e4 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:28.291859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c435b8-8875-46bd-993c-a6996d82aa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:28.292698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c435b8-8875-46bd-993c-a6996d82aa44 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:28.292976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c435b8-8875-46bd-993c-a6996d82aa44 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:28.293248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c435b8-8875-46bd-993c-a6996d82aa44 None admin] PATH_INFO: `/v3/projects/d8164b89432449f08d1bc8aaa877452b` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:28.294549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c435b8-8875-46bd-993c-a6996d82aa44 None admin] RBAC: Authorizing `identity:get_project(project_id=d8164b89432449f08d1bc8aaa877452b)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:28.295966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c435b8-8875-46bd-993c-a6996d82aa44 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:28.297668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 567/1129] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:28 2022] GET /identity/v3/projects/d8164b89432449f08d1bc8aaa877452b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:28.303647 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3754b-d4c2-48f5-b8f9-b45164d229fb None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:28.399368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3754b-d4c2-48f5-b8f9-b45164d229fb None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:28.431455 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170433c8-f800-446d-9127-946bc08b5d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:28.432095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170433c8-f800-446d-9127-946bc08b5d20 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:28.432379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170433c8-f800-446d-9127-946bc08b5d20 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:28.432658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170433c8-f800-446d-9127-946bc08b5d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:28.433480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170433c8-f800-446d-9127-946bc08b5d20 None admin] RBAC: Adding query filter params (role.id=b127031d393048d7ae9a35015113e82a, scope.project.id=d8164b89432449f08d1bc8aaa877452b, user.id=5148eeb539b049b8a03158acc861dbce) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:28.433824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170433c8-f800-446d-9127-946bc08b5d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:28.435784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170433c8-f800-446d-9127-946bc08b5d20 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:28.441410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 563/1130] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:41:28 2022] GET /identity/v3/role_assignments?user.id=5148eeb539b049b8a03158acc861dbce&scope.project.id=d8164b89432449f08d1bc8aaa877452b&role.id=b127031d393048d7ae9a35015113e82a => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:30.494108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b2676-dc69-4cd4-b83a-963e3626bba4 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:30.494656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b2676-dc69-4cd4-b83a-963e3626bba4 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:30.494656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b2676-dc69-4cd4-b83a-963e3626bba4 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:30.496019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 568/1131] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:30.501162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff70ee-2f44-47df-b6f8-58e577aa51e4 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:30.501608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff70ee-2f44-47df-b6f8-58e577aa51e4 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:30.501753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff70ee-2f44-47df-b6f8-58e577aa51e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:30.506691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:30.520378 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-00ff70ee-2f44-47df-b6f8-58e577aa51e4 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:30.521540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-00ff70ee-2f44-47df-b6f8-58e577aa51e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:30.587850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 564/1132] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:30.612844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b2676-dc69-4cd4-b83a-963e3626bba4 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:30.711523 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b2676-dc69-4cd4-b83a-963e3626bba4 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:30.744313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb0110-94fd-4ec2-8b5c-7c80b34878e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:30.745032 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb0110-94fd-4ec2-8b5c-7c80b34878e9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:30.745314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb0110-94fd-4ec2-8b5c-7c80b34878e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:30.745616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb0110-94fd-4ec2-8b5c-7c80b34878e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:30.746875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb0110-94fd-4ec2-8b5c-7c80b34878e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:30.748319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb0110-94fd-4ec2-8b5c-7c80b34878e9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:30.749956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 569/1133] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:30.755605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff70ee-2f44-47df-b6f8-58e577aa51e4 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:30.850697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff70ee-2f44-47df-b6f8-58e577aa51e4 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:30.878639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:30.883937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b453fd-25ff-4e41-beba-d47207103cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:30.884573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b453fd-25ff-4e41-beba-d47207103cbc None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:30.884848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b453fd-25ff-4e41-beba-d47207103cbc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:30.885115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b453fd-25ff-4e41-beba-d47207103cbc None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:30.885844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b453fd-25ff-4e41-beba-d47207103cbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:30.888340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b453fd-25ff-4e41-beba-d47207103cbc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:30.912910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 565/1134] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:41:30 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:41:32.991573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a09178-5907-44b6-b546-f9e94bfe6509 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:32.992119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a09178-5907-44b6-b546-f9e94bfe6509 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:32.992119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a09178-5907-44b6-b546-f9e94bfe6509 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:32.993470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 570/1135] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:32.998765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbfd01-4bd2-44ac-b748-725d2143f8a8 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:32.999180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbfd01-4bd2-44ac-b748-725d2143f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:32.999292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbfd01-4bd2-44ac-b748-725d2143f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:33.015891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7bdbfd01-4bd2-44ac-b748-725d2143f8a8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:33.017061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7bdbfd01-4bd2-44ac-b748-725d2143f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:33.081224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 566/1136] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:33.106263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a09178-5907-44b6-b546-f9e94bfe6509 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:33.115003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:33.209490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a09178-5907-44b6-b546-f9e94bfe6509 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:33.242504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda9e49-3ba9-4253-9ecf-627ea57c89c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:33.243221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda9e49-3ba9-4253-9ecf-627ea57c89c1 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:33.243497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda9e49-3ba9-4253-9ecf-627ea57c89c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:33.243764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda9e49-3ba9-4253-9ecf-627ea57c89c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:33.244918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda9e49-3ba9-4253-9ecf-627ea57c89c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:33.246477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda9e49-3ba9-4253-9ecf-627ea57c89c1 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:33.248579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 571/1137] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:41:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:33.254359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbfd01-4bd2-44ac-b748-725d2143f8a8 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:33.348166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbfd01-4bd2-44ac-b748-725d2143f8a8 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:33.362969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:33.383149 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55576664-3a3a-4368-8e24-87587d897cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:33.383804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55576664-3a3a-4368-8e24-87587d897cd6 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:33.384084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55576664-3a3a-4368-8e24-87587d897cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:33.384358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55576664-3a3a-4368-8e24-87587d897cd6 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:33.385065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55576664-3a3a-4368-8e24-87587d897cd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:33.386560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55576664-3a3a-4368-8e24-87587d897cd6 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:33.406718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 567/1138] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:41:33 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:41:35.509058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90898f7-e8d4-49c7-8c85-973572f38105 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:35.509625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90898f7-e8d4-49c7-8c85-973572f38105 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:35.509625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90898f7-e8d4-49c7-8c85-973572f38105 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:35.511013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 572/1139] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:35.517232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b3be0-00dc-4e75-894e-1f17ed01c234 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:35.517686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b3be0-00dc-4e75-894e-1f17ed01c234 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:35.517813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b3be0-00dc-4e75-894e-1f17ed01c234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:35.535396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-261b3be0-00dc-4e75-894e-1f17ed01c234 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:35.536557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-261b3be0-00dc-4e75-894e-1f17ed01c234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:35.603270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 568/1140] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:35.628175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90898f7-e8d4-49c7-8c85-973572f38105 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:35.702202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:35.727885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90898f7-e8d4-49c7-8c85-973572f38105 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:35.761375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce978b-3263-4ff7-8568-727c5c7e58a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:35.762243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce978b-3263-4ff7-8568-727c5c7e58a2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:35.762527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce978b-3263-4ff7-8568-727c5c7e58a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:35.762814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce978b-3263-4ff7-8568-727c5c7e58a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:35.772647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce978b-3263-4ff7-8568-727c5c7e58a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-87ce978b-3263-4ff7-8568-727c5c7e58a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:35.780877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:35.782135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:35.783193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 573/1141] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:41:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:35.788390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b3be0-00dc-4e75-894e-1f17ed01c234 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:35.883374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:35.886460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b3be0-00dc-4e75-894e-1f17ed01c234 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:35.921974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc12074-fc3c-4699-860c-d826cd0b78be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:35.922628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc12074-fc3c-4699-860c-d826cd0b78be None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:35.922909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc12074-fc3c-4699-860c-d826cd0b78be None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:35.923186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc12074-fc3c-4699-860c-d826cd0b78be None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:35.923925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc12074-fc3c-4699-860c-d826cd0b78be None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:35.924230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc12074-fc3c-4699-860c-d826cd0b78be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:35.925621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc12074-fc3c-4699-860c-d826cd0b78be None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:35.933410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 569/1142] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:41:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:35.939239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce978b-3263-4ff7-8568-727c5c7e58a2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:36.035759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce978b-3263-4ff7-8568-727c5c7e58a2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:36.068541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34763c-3bba-4836-9717-336bce59d9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:36.069388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34763c-3bba-4836-9717-336bce59d9b9 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:36.069690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34763c-3bba-4836-9717-336bce59d9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:36.069960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34763c-3bba-4836-9717-336bce59d9b9 None admin] PATH_INFO: `/v3/users/6464b4c4103944c0802070d89994474f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:36.071807 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:36.083124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34763c-3bba-4836-9717-336bce59d9b9 None admin] RBAC: Authorizing `identity:get_user(user_id=6464b4c4103944c0802070d89994474f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:36.084560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34763c-3bba-4836-9717-336bce59d9b9 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:36.086231 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 574/1143] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:35 2022] GET /identity/v3/users/6464b4c4103944c0802070d89994474f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:36.091999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc12074-fc3c-4699-860c-d826cd0b78be None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:36.206424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc12074-fc3c-4699-860c-d826cd0b78be None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:36.238561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953111de-f912-4e90-8183-c0ca658f4b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:36.239201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953111de-f912-4e90-8183-c0ca658f4b38 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:36.239491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953111de-f912-4e90-8183-c0ca658f4b38 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:36.239766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953111de-f912-4e90-8183-c0ca658f4b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:36.268622 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:36.305654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953111de-f912-4e90-8183-c0ca658f4b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:36.307056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953111de-f912-4e90-8183-c0ca658f4b38 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:36.401598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 570/1144] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:41:36.408256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34763c-3bba-4836-9717-336bce59d9b9 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:36.506194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34763c-3bba-4836-9717-336bce59d9b9 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:36.540593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705ae05-beae-43f2-9ea1-60358c843526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:36.541433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705ae05-beae-43f2-9ea1-60358c843526 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:36.541738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705ae05-beae-43f2-9ea1-60358c843526 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:36.542004 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705ae05-beae-43f2-9ea1-60358c843526 None admin] PATH_INFO: `/v3/projects/6b148d65117946ceb845b095560692d6` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:36.543160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f705ae05-beae-43f2-9ea1-60358c843526 None admin] RBAC: Authorizing `identity:get_project(project_id=6b148d65117946ceb845b095560692d6)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:36.545795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f705ae05-beae-43f2-9ea1-60358c843526 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:36.547641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 575/1145] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:36 2022] GET /identity/v3/projects/6b148d65117946ceb845b095560692d6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:36.553681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953111de-f912-4e90-8183-c0ca658f4b38 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:36.613129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:36.649404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953111de-f912-4e90-8183-c0ca658f4b38 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:36.682666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efca135-f7b5-4c6c-9796-de219e252a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:36.683307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efca135-f7b5-4c6c-9796-de219e252a40 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:36.683586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efca135-f7b5-4c6c-9796-de219e252a40 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:36.683858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efca135-f7b5-4c6c-9796-de219e252a40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:36.684682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efca135-f7b5-4c6c-9796-de219e252a40 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=6b148d65117946ceb845b095560692d6, user.id=6464b4c4103944c0802070d89994474f) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:36.685025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efca135-f7b5-4c6c-9796-de219e252a40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:36.686568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efca135-f7b5-4c6c-9796-de219e252a40 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:36.691837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 571/1146] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:41:36 2022] GET /identity/v3/role_assignments?user.id=6464b4c4103944c0802070d89994474f&scope.project.id=6b148d65117946ceb845b095560692d6&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:38.717534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeea5a-679d-4efd-9222-c3049b4e4c47 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:38.718098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeea5a-679d-4efd-9222-c3049b4e4c47 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:38.718098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeea5a-679d-4efd-9222-c3049b4e4c47 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:38.719452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 576/1147] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:38.724549 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d0b74-b921-4ac2-9650-23d7975eae45 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:38.724916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d0b74-b921-4ac2-9650-23d7975eae45 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:38.725104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d0b74-b921-4ac2-9650-23d7975eae45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:38.742594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ba8d0b74-b921-4ac2-9650-23d7975eae45 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:38.743780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ba8d0b74-b921-4ac2-9650-23d7975eae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:38.808437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 572/1148] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:38.830996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeea5a-679d-4efd-9222-c3049b4e4c47 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:38.889159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:38.929889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeea5a-679d-4efd-9222-c3049b4e4c47 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:38.962912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d79a38-022f-40fc-8c3f-d85e1b9007a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:38.963746 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d79a38-022f-40fc-8c3f-d85e1b9007a0 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:38.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d79a38-022f-40fc-8c3f-d85e1b9007a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:38.964325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d79a38-022f-40fc-8c3f-d85e1b9007a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:38.974957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d79a38-022f-40fc-8c3f-d85e1b9007a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-c4d79a38-022f-40fc-8c3f-d85e1b9007a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:38.982777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:38.983832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:38.985054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 577/1149] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:41:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:38.990180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d0b74-b921-4ac2-9650-23d7975eae45 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:39.043572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:39.085446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d0b74-b921-4ac2-9650-23d7975eae45 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:39.117734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51b5f6-a527-4f6e-adc2-ecf617d70369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:39.118383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51b5f6-a527-4f6e-adc2-ecf617d70369 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:39.118659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51b5f6-a527-4f6e-adc2-ecf617d70369 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:39.118930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51b5f6-a527-4f6e-adc2-ecf617d70369 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:39.119658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51b5f6-a527-4f6e-adc2-ecf617d70369 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:39.119957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51b5f6-a527-4f6e-adc2-ecf617d70369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:39.121303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51b5f6-a527-4f6e-adc2-ecf617d70369 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:39.129806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 573/1150] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:41:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:39.135344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d79a38-022f-40fc-8c3f-d85e1b9007a0 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:39.233141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d79a38-022f-40fc-8c3f-d85e1b9007a0 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:39.252729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:39.266850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab52983-cfd3-4892-84e0-1213b46b0815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:39.267702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab52983-cfd3-4892-84e0-1213b46b0815 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:39.267984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab52983-cfd3-4892-84e0-1213b46b0815 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:39.268257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab52983-cfd3-4892-84e0-1213b46b0815 None admin] PATH_INFO: `/v3/users/6464b4c4103944c0802070d89994474f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:39.269433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab52983-cfd3-4892-84e0-1213b46b0815 None admin] RBAC: Authorizing `identity:get_user(user_id=6464b4c4103944c0802070d89994474f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:39.270892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab52983-cfd3-4892-84e0-1213b46b0815 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:39.272511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 578/1151] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:39 2022] GET /identity/v3/users/6464b4c4103944c0802070d89994474f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:39.278337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51b5f6-a527-4f6e-adc2-ecf617d70369 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:39.375351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51b5f6-a527-4f6e-adc2-ecf617d70369 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:39.391200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:39.408849 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf1198-1188-4861-9eed-ce91ee12e109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:39.409999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf1198-1188-4861-9eed-ce91ee12e109 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:39.410278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf1198-1188-4861-9eed-ce91ee12e109 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:39.410548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf1198-1188-4861-9eed-ce91ee12e109 None admin] PATH_INFO: `/v3/projects/6b148d65117946ceb845b095560692d6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:39.411710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf1198-1188-4861-9eed-ce91ee12e109 None admin] RBAC: Authorizing `identity:get_project(project_id=6b148d65117946ceb845b095560692d6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:39.413124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf1198-1188-4861-9eed-ce91ee12e109 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:39.414813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 574/1152] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:39 2022] GET /identity/v3/projects/6b148d65117946ceb845b095560692d6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:39.421155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab52983-cfd3-4892-84e0-1213b46b0815 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:39.517809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab52983-cfd3-4892-84e0-1213b46b0815 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:39.553793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a63043-5282-4834-ae13-64062dabec33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:39.554670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a63043-5282-4834-ae13-64062dabec33 None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:39.554952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a63043-5282-4834-ae13-64062dabec33 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:39.555224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a63043-5282-4834-ae13-64062dabec33 None admin] PATH_INFO: `/v3/projects/6b148d65117946ceb845b095560692d6/users/6464b4c4103944c0802070d89994474f/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:39.557163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a63043-5282-4834-ae13-64062dabec33 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b148d65117946ceb845b095560692d6, user_id=6464b4c4103944c0802070d89994474f, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:39.558696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a63043-5282-4834-ae13-64062dabec33 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:39.571690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 579/1153] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:41:39 2022] PUT /identity/v3/projects/6b148d65117946ceb845b095560692d6/users/6464b4c4103944c0802070d89994474f/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:41:41.593046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73311fea-e572-454e-9180-71765d9ee79d None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:41.593572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73311fea-e572-454e-9180-71765d9ee79d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:41.593572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73311fea-e572-454e-9180-71765d9ee79d None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:41.594975 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 575/1154] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:41.600300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3491-dc5c-4146-be16-c8180b236992 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:41.600707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3491-dc5c-4146-be16-c8180b236992 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:41.600858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3491-dc5c-4146-be16-c8180b236992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:41.618314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-907f3491-dc5c-4146-be16-c8180b236992 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:41.619497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-907f3491-dc5c-4146-be16-c8180b236992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:41.662342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:41.686498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 580/1155] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:41.709711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73311fea-e572-454e-9180-71765d9ee79d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:41.808077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73311fea-e572-454e-9180-71765d9ee79d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:41.842588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3132f1-c77d-427b-a9de-0e74ce7f80c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:41.843374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3132f1-c77d-427b-a9de-0e74ce7f80c3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:41.843650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3132f1-c77d-427b-a9de-0e74ce7f80c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:41.843919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3132f1-c77d-427b-a9de-0e74ce7f80c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:41.853633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3132f1-c77d-427b-a9de-0e74ce7f80c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-9f3132f1-c77d-427b-a9de-0e74ce7f80c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:41.862354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:41.863639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:41.865568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:41.867166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 576/1156] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:41:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:41.878809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f3491-dc5c-4146-be16-c8180b236992 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:41.975871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f3491-dc5c-4146-be16-c8180b236992 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:42.008569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705a324-bdd2-4648-bf7c-76e931fad883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:42.009214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705a324-bdd2-4648-bf7c-76e931fad883 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:42.009495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705a324-bdd2-4648-bf7c-76e931fad883 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:42.009795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705a324-bdd2-4648-bf7c-76e931fad883 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:42.010534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705a324-bdd2-4648-bf7c-76e931fad883 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:42.010835 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705a324-bdd2-4648-bf7c-76e931fad883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:42.012195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705a324-bdd2-4648-bf7c-76e931fad883 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:42.020586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 581/1157] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:41:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:42.026394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3132f1-c77d-427b-a9de-0e74ce7f80c3 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:42.073463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:42.125263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3132f1-c77d-427b-a9de-0e74ce7f80c3 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:42.157853 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1621d9-5cb2-4487-81e5-dbf5ed0ca77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:42.158732 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1621d9-5cb2-4487-81e5-dbf5ed0ca77f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:42.159284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1621d9-5cb2-4487-81e5-dbf5ed0ca77f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:42.159560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1621d9-5cb2-4487-81e5-dbf5ed0ca77f None admin] PATH_INFO: `/v3/users/6464b4c4103944c0802070d89994474f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:42.160888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1621d9-5cb2-4487-81e5-dbf5ed0ca77f None admin] RBAC: Authorizing `identity:get_user(user_id=6464b4c4103944c0802070d89994474f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:42.162356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1621d9-5cb2-4487-81e5-dbf5ed0ca77f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:42.164028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 577/1158] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:42 2022] GET /identity/v3/users/6464b4c4103944c0802070d89994474f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:42.171007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705a324-bdd2-4648-bf7c-76e931fad883 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:42.212098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:42.269065 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705a324-bdd2-4648-bf7c-76e931fad883 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:42.302110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345cf45-587e-41e8-8698-daee36cce1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:42.302740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345cf45-587e-41e8-8698-daee36cce1bb None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:42.303022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345cf45-587e-41e8-8698-daee36cce1bb None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:42.303297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345cf45-587e-41e8-8698-daee36cce1bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:42.368338 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345cf45-587e-41e8-8698-daee36cce1bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:42.369754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345cf45-587e-41e8-8698-daee36cce1bb None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:42.408252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:42.465077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 582/1159] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:41:42.471697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1621d9-5cb2-4487-81e5-dbf5ed0ca77f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:42.571288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1621d9-5cb2-4487-81e5-dbf5ed0ca77f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:42.584562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:42.605811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d2d83-e78f-413f-820c-672a2edd3951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:42.607308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d2d83-e78f-413f-820c-672a2edd3951 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:42.607689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d2d83-e78f-413f-820c-672a2edd3951 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:42.607970 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d2d83-e78f-413f-820c-672a2edd3951 None admin] PATH_INFO: `/v3/projects/6b148d65117946ceb845b095560692d6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:42.609156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d2d83-e78f-413f-820c-672a2edd3951 None admin] RBAC: Authorizing `identity:get_project(project_id=6b148d65117946ceb845b095560692d6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:42.610606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d2d83-e78f-413f-820c-672a2edd3951 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:42.612289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 578/1160] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:42 2022] GET /identity/v3/projects/6b148d65117946ceb845b095560692d6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:42.618413 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345cf45-587e-41e8-8698-daee36cce1bb None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:42.714071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345cf45-587e-41e8-8698-daee36cce1bb None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:42.747143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78374c29-ad69-4e12-b14a-8477de8e2dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:42.747775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78374c29-ad69-4e12-b14a-8477de8e2dae None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:42.748057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78374c29-ad69-4e12-b14a-8477de8e2dae None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:42.748331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78374c29-ad69-4e12-b14a-8477de8e2dae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:42.749143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78374c29-ad69-4e12-b14a-8477de8e2dae None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=6b148d65117946ceb845b095560692d6, user.id=6464b4c4103944c0802070d89994474f) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:42.749452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78374c29-ad69-4e12-b14a-8477de8e2dae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:42.750901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78374c29-ad69-4e12-b14a-8477de8e2dae None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:42.756554 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 583/1161] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:41:42 2022] GET /identity/v3/role_assignments?user.id=6464b4c4103944c0802070d89994474f&scope.project.id=6b148d65117946ceb845b095560692d6&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:44.821710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb95983-5947-489a-887b-c1d3c0af5811 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:44.822213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb95983-5947-489a-887b-c1d3c0af5811 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:44.822304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb95983-5947-489a-887b-c1d3c0af5811 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:44.823624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 579/1162] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:44.829258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e09a-8bae-4f67-bd6f-b6ba7e6d5c53 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:44.829701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e09a-8bae-4f67-bd6f-b6ba7e6d5c53 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:44.829841 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e09a-8bae-4f67-bd6f-b6ba7e6d5c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:44.835760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:44.846845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-cad9e09a-8bae-4f67-bd6f-b6ba7e6d5c53 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:44.848028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-cad9e09a-8bae-4f67-bd6f-b6ba7e6d5c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:44.914166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 584/1163] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:44.937684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb95983-5947-489a-887b-c1d3c0af5811 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:45.036219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb95983-5947-489a-887b-c1d3c0af5811 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:45.069542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31758ead-35d4-4570-ad84-9e38910f3dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:45.070215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31758ead-35d4-4570-ad84-9e38910f3dce None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:45.070498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31758ead-35d4-4570-ad84-9e38910f3dce None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:45.070773 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31758ead-35d4-4570-ad84-9e38910f3dce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:45.117325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:45.136103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31758ead-35d4-4570-ad84-9e38910f3dce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:45.137574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31758ead-35d4-4570-ad84-9e38910f3dce None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:45.233750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 580/1164] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:41:45.239684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9e09a-8bae-4f67-bd6f-b6ba7e6d5c53 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:45.335491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9e09a-8bae-4f67-bd6f-b6ba7e6d5c53 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:45.368805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639a262-4aa7-4eba-b181-204631889963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:45.369522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639a262-4aa7-4eba-b181-204631889963 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:45.369828 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639a262-4aa7-4eba-b181-204631889963 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:45.370099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639a262-4aa7-4eba-b181-204631889963 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:45.372232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:45.377941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639a262-4aa7-4eba-b181-204631889963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:45.379572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639a262-4aa7-4eba-b181-204631889963 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-a639a262-4aa7-4eba-b181-204631889963 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:45.393610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:45.394791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:41:45.395631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 585/1165] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 1 19:41:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:45.400633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31758ead-35d4-4570-ad84-9e38910f3dce None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:45.480870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:45.499000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31758ead-35d4-4570-ad84-9e38910f3dce None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:45.532738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988dbd6-f677-4c32-971d-7fbbe0f11afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:45.533390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988dbd6-f677-4c32-971d-7fbbe0f11afc None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:45.533699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988dbd6-f677-4c32-971d-7fbbe0f11afc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:45.534236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988dbd6-f677-4c32-971d-7fbbe0f11afc None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:45.534947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988dbd6-f677-4c32-971d-7fbbe0f11afc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:45.535251 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988dbd6-f677-4c32-971d-7fbbe0f11afc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:45.536603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988dbd6-f677-4c32-971d-7fbbe0f11afc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:45.543555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 581/1166] 10.222.0.66 () {56 vars in 1115 bytes} [Fri Jul 1 19:41:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:47.585162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b364760-eaa5-493b-b689-b1522aae2b60 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:47.585770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b364760-eaa5-493b-b689-b1522aae2b60 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:47.585770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b364760-eaa5-493b-b689-b1522aae2b60 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:47.587099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 586/1167] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:47.593051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2b8ab-3e38-44c8-ae0d-8e0eb1ec3cda None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:47.593481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2b8ab-3e38-44c8-ae0d-8e0eb1ec3cda None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:47.593614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2b8ab-3e38-44c8-ae0d-8e0eb1ec3cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:47.610307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c8b2b8ab-3e38-44c8-ae0d-8e0eb1ec3cda None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:47.611474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c8b2b8ab-3e38-44c8-ae0d-8e0eb1ec3cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:47.674946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 582/1168] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:47.697960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b364760-eaa5-493b-b689-b1522aae2b60 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:47.797542 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b364760-eaa5-493b-b689-b1522aae2b60 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:47.829996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93235a-7a06-4f2a-bdb0-755385db5496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:47.830638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93235a-7a06-4f2a-bdb0-755385db5496 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:47.830913 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93235a-7a06-4f2a-bdb0-755385db5496 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:47.831179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93235a-7a06-4f2a-bdb0-755385db5496 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:47.831922 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93235a-7a06-4f2a-bdb0-755385db5496 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:47.833270 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93235a-7a06-4f2a-bdb0-755385db5496 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:47.848013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 587/1169] 10.222.0.66 () {60 vars in 1138 bytes} [Fri Jul 1 19:41:47 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:41:49.911082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a366ab-0025-458d-b22e-d9d7d05d8d02 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:49.911634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a366ab-0025-458d-b22e-d9d7d05d8d02 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:49.911634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a366ab-0025-458d-b22e-d9d7d05d8d02 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:49.912999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 583/1170] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:49.918296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32ce63-eb42-4d1b-aa80-9d519b8f236d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:49.918687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32ce63-eb42-4d1b-aa80-9d519b8f236d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:49.918897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32ce63-eb42-4d1b-aa80-9d519b8f236d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:49.936217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-0a32ce63-eb42-4d1b-aa80-9d519b8f236d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:49.937411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0a32ce63-eb42-4d1b-aa80-9d519b8f236d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:49.986677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:50.004907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 588/1171] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:50.029062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a366ab-0025-458d-b22e-d9d7d05d8d02 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:50.104380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:50.128443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a366ab-0025-458d-b22e-d9d7d05d8d02 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:50.161657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722ecae-9234-4f03-986e-30631f6cac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:50.162458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722ecae-9234-4f03-986e-30631f6cac78 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:50.162739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722ecae-9234-4f03-986e-30631f6cac78 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:50.163012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722ecae-9234-4f03-986e-30631f6cac78 None admin] PATH_INFO: `/v3/domains/d7024e57c5084dfdb0f93d3ca12f4453` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:50.173807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722ecae-9234-4f03-986e-30631f6cac78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d7024e57c5084dfdb0f93d3ca12f4453)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:50.175254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722ecae-9234-4f03-986e-30631f6cac78 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:50.176855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 584/1172] 10.222.0.66 () {56 vars in 1183 bytes} [Fri Jul 1 19:41:50 2022] GET /identity/v3/domains/d7024e57c5084dfdb0f93d3ca12f4453 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:50.182749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32ce63-eb42-4d1b-aa80-9d519b8f236d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:50.279948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32ce63-eb42-4d1b-aa80-9d519b8f236d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:50.312740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe24a1-7595-432a-932e-c5b318117581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:50.313385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe24a1-7595-432a-932e-c5b318117581 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:50.313697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe24a1-7595-432a-932e-c5b318117581 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:50.313976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe24a1-7595-432a-932e-c5b318117581 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:50.314677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe24a1-7595-432a-932e-c5b318117581 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:50.316098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe24a1-7595-432a-932e-c5b318117581 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:50.340920 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 589/1173] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:41:50 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:41:52.413085 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d356-1c64-423b-856a-935cc657f504 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:52.413682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d356-1c64-423b-856a-935cc657f504 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:52.413682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d356-1c64-423b-856a-935cc657f504 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:52.415017 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 585/1174] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:52.421211 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9feeae-6136-4766-8c4f-1e6b4fdd2b12 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:52.421620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9feeae-6136-4766-8c4f-1e6b4fdd2b12 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:52.421793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9feeae-6136-4766-8c4f-1e6b4fdd2b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:52.438017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-2e9feeae-6136-4766-8c4f-1e6b4fdd2b12 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:52.439177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2e9feeae-6136-4766-8c4f-1e6b4fdd2b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:52.467490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:52.504308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 590/1175] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:41:52.530221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d356-1c64-423b-856a-935cc657f504 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:52.629929 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d356-1c64-423b-856a-935cc657f504 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:52.662284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19680d-8827-49ce-81fa-55740b122b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:52.663192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19680d-8827-49ce-81fa-55740b122b00 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:52.663413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19680d-8827-49ce-81fa-55740b122b00 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:52.663682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19680d-8827-49ce-81fa-55740b122b00 None admin] PATH_INFO: `/v3/domains/d7024e57c5084dfdb0f93d3ca12f4453` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:52.664832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19680d-8827-49ce-81fa-55740b122b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d7024e57c5084dfdb0f93d3ca12f4453)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:52.666303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19680d-8827-49ce-81fa-55740b122b00 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:52.667897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 586/1176] 10.222.0.66 () {56 vars in 1183 bytes} [Fri Jul 1 19:41:52 2022] GET /identity/v3/domains/d7024e57c5084dfdb0f93d3ca12f4453 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:52.674423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9feeae-6136-4766-8c4f-1e6b4fdd2b12 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:52.768799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9feeae-6136-4766-8c4f-1e6b4fdd2b12 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:52.801533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649a17d-6117-4dc1-b492-88f5a71ae70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:52.802204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649a17d-6117-4dc1-b492-88f5a71ae70a None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:52.802482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649a17d-6117-4dc1-b492-88f5a71ae70a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:52.802749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649a17d-6117-4dc1-b492-88f5a71ae70a None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:52.803436 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649a17d-6117-4dc1-b492-88f5a71ae70a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:52.804847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649a17d-6117-4dc1-b492-88f5a71ae70a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:52.824119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 591/1177] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:41:52 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:41:54.923278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753eb85-bb55-4982-9492-4db3616a61f0 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:54.923800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753eb85-bb55-4982-9492-4db3616a61f0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:54.923892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753eb85-bb55-4982-9492-4db3616a61f0 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:54.925190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 587/1178] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:54.930386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fdfa-de92-44f9-a5a6-6da1773b526b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:54.930774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fdfa-de92-44f9-a5a6-6da1773b526b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:54.930936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482fdfa-de92-44f9-a5a6-6da1773b526b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:54.947124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-6482fdfa-de92-44f9-a5a6-6da1773b526b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:54.948302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6482fdfa-de92-44f9-a5a6-6da1773b526b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:54.965052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:55.015742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 592/1179] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:55.039585 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753eb85-bb55-4982-9492-4db3616a61f0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:55.043978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:55.139160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753eb85-bb55-4982-9492-4db3616a61f0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:55.173515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fe8fd-ba43-4724-9fb4-8baa61e1efd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:55.174257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fe8fd-ba43-4724-9fb4-8baa61e1efd5 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:55.174534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fe8fd-ba43-4724-9fb4-8baa61e1efd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:55.174801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fe8fd-ba43-4724-9fb4-8baa61e1efd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:55.184594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5fe8fd-ba43-4724-9fb4-8baa61e1efd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-7e5fe8fd-ba43-4724-9fb4-8baa61e1efd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:55.192566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:55.193777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:55.194826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 588/1180] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:41:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:55.199658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482fdfa-de92-44f9-a5a6-6da1773b526b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:55.296925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482fdfa-de92-44f9-a5a6-6da1773b526b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:55.329477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b0a32-debe-4312-9ac8-27d0da88f696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:55.330178 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b0a32-debe-4312-9ac8-27d0da88f696 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:55.330458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b0a32-debe-4312-9ac8-27d0da88f696 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:55.330731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b0a32-debe-4312-9ac8-27d0da88f696 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:55.331469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b0a32-debe-4312-9ac8-27d0da88f696 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:55.331776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b0a32-debe-4312-9ac8-27d0da88f696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:55.333141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b0a32-debe-4312-9ac8-27d0da88f696 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:55.341597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 593/1181] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:55.347207 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fe8fd-ba43-4724-9fb4-8baa61e1efd5 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:55.407604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:55.444672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fe8fd-ba43-4724-9fb4-8baa61e1efd5 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:55.477044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1b57b-2865-4d9b-aaf9-ee81da9fb4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:55.477923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1b57b-2865-4d9b-aaf9-ee81da9fb4dd None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:55.478199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1b57b-2865-4d9b-aaf9-ee81da9fb4dd None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:55.478468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1b57b-2865-4d9b-aaf9-ee81da9fb4dd None admin] PATH_INFO: `/v3/users/ba9468feae5144c78f961dfe239cf73b` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:55.491058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1b57b-2865-4d9b-aaf9-ee81da9fb4dd None admin] RBAC: Authorizing `identity:get_user(user_id=ba9468feae5144c78f961dfe239cf73b)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:55.492482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1b57b-2865-4d9b-aaf9-ee81da9fb4dd None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:55.494114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 589/1182] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:55 2022] GET /identity/v3/users/ba9468feae5144c78f961dfe239cf73b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:55.499850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b0a32-debe-4312-9ac8-27d0da88f696 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:55.516101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:55.600944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b0a32-debe-4312-9ac8-27d0da88f696 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:55.634037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a335-ebf9-449a-910e-467ae865f982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:55.634673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a335-ebf9-449a-910e-467ae865f982 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:55.634954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a335-ebf9-449a-910e-467ae865f982 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:55.635227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a335-ebf9-449a-910e-467ae865f982 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:55.700850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1a335-ebf9-449a-910e-467ae865f982 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:55.702251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1a335-ebf9-449a-910e-467ae865f982 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:55.715942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:55.797135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 594/1183] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:41:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:41:55.803017 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1b57b-2865-4d9b-aaf9-ee81da9fb4dd None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:55.898846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1b57b-2865-4d9b-aaf9-ee81da9fb4dd None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:55.921682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:55.933189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524465d-5647-48f1-96e0-15425f94e385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:55.934137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524465d-5647-48f1-96e0-15425f94e385 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:55.934421 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524465d-5647-48f1-96e0-15425f94e385 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:55.934697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524465d-5647-48f1-96e0-15425f94e385 None admin] PATH_INFO: `/v3/projects/46185aeb22dd418fa143b0ec720e8231` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:55.935913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524465d-5647-48f1-96e0-15425f94e385 None admin] RBAC: Authorizing `identity:get_project(project_id=46185aeb22dd418fa143b0ec720e8231)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:55.938232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524465d-5647-48f1-96e0-15425f94e385 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:55.941071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 590/1184] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:55 2022] GET /identity/v3/projects/46185aeb22dd418fa143b0ec720e8231 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:55.947680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a335-ebf9-449a-910e-467ae865f982 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:56.044988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a335-ebf9-449a-910e-467ae865f982 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:56.059420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:56.079150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ee230-0176-4ae4-bcea-e0a7cdbaaf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:56.079788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ee230-0176-4ae4-bcea-e0a7cdbaaf94 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:56.080070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ee230-0176-4ae4-bcea-e0a7cdbaaf94 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:56.080343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ee230-0176-4ae4-bcea-e0a7cdbaaf94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:56.081159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ee230-0176-4ae4-bcea-e0a7cdbaaf94 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=46185aeb22dd418fa143b0ec720e8231, user.id=ba9468feae5144c78f961dfe239cf73b) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:56.081467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ee230-0176-4ae4-bcea-e0a7cdbaaf94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:56.082911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ee230-0176-4ae4-bcea-e0a7cdbaaf94 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:56.088174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 595/1185] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:41:55 2022] GET /identity/v3/role_assignments?user.id=ba9468feae5144c78f961dfe239cf73b&scope.project.id=46185aeb22dd418fa143b0ec720e8231&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:58.116407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86674bd-7fee-461b-8b94-ba0af7d4306d None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:58.116935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86674bd-7fee-461b-8b94-ba0af7d4306d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:58.116935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86674bd-7fee-461b-8b94-ba0af7d4306d None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:58.118346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 591/1186] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:41:58.123638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811f000-af90-408a-8255-e25894379d61 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:58.124008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811f000-af90-408a-8255-e25894379d61 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:58.124224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811f000-af90-408a-8255-e25894379d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:58.140598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-4811f000-af90-408a-8255-e25894379d61 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:41:58.141819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4811f000-af90-408a-8255-e25894379d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:41:58.206716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 596/1187] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:41:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:41:58.229688 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86674bd-7fee-461b-8b94-ba0af7d4306d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:58.328427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86674bd-7fee-461b-8b94-ba0af7d4306d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:58.361329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a87952-2433-4e14-8eec-82bd824407e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:58.362132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a87952-2433-4e14-8eec-82bd824407e6 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:58.362417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a87952-2433-4e14-8eec-82bd824407e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:58.362687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a87952-2433-4e14-8eec-82bd824407e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:58.372864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a87952-2433-4e14-8eec-82bd824407e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-e8a87952-2433-4e14-8eec-82bd824407e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:41:58.380693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:41:58.381805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:41:58.382911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 592/1188] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:41:58.388761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811f000-af90-408a-8255-e25894379d61 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:58.486287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811f000-af90-408a-8255-e25894379d61 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:58.495073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:58.520883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5e207-96fd-42a0-82a9-09f57e1fbb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:58.521545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5e207-96fd-42a0-82a9-09f57e1fbb2c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:58.521874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5e207-96fd-42a0-82a9-09f57e1fbb2c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:58.522146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5e207-96fd-42a0-82a9-09f57e1fbb2c None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:58.522889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5e207-96fd-42a0-82a9-09f57e1fbb2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:41:58.523193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5e207-96fd-42a0-82a9-09f57e1fbb2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:58.524557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5e207-96fd-42a0-82a9-09f57e1fbb2c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:58.533126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 597/1189] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:58.539212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a87952-2433-4e14-8eec-82bd824407e6 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:58.587713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:58.635683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a87952-2433-4e14-8eec-82bd824407e6 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:58.668905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab6825-47f9-4abd-adb7-07efe3b84c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:58.669801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6825-47f9-4abd-adb7-07efe3b84c33 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:58.670082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6825-47f9-4abd-adb7-07efe3b84c33 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:58.670355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6825-47f9-4abd-adb7-07efe3b84c33 None admin] PATH_INFO: `/v3/users/ba9468feae5144c78f961dfe239cf73b` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:58.671525 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab6825-47f9-4abd-adb7-07efe3b84c33 None admin] RBAC: Authorizing `identity:get_user(user_id=ba9468feae5144c78f961dfe239cf73b)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:58.672961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab6825-47f9-4abd-adb7-07efe3b84c33 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:58.674598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 593/1190] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:41:58 2022] GET /identity/v3/users/ba9468feae5144c78f961dfe239cf73b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:41:58.680494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5e207-96fd-42a0-82a9-09f57e1fbb2c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:58.778184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5e207-96fd-42a0-82a9-09f57e1fbb2c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:58.810638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454031f2-e34f-4530-b365-17948d342bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:58.811488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454031f2-e34f-4530-b365-17948d342bd0 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:58.811770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454031f2-e34f-4530-b365-17948d342bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:58.812046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454031f2-e34f-4530-b365-17948d342bd0 None admin] PATH_INFO: `/v3/projects/46185aeb22dd418fa143b0ec720e8231` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:58.813216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454031f2-e34f-4530-b365-17948d342bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=46185aeb22dd418fa143b0ec720e8231)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:58.814831 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454031f2-e34f-4530-b365-17948d342bd0 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:58.816515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 598/1191] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:41:58 2022] GET /identity/v3/projects/46185aeb22dd418fa143b0ec720e8231 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:41:58.822333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab6825-47f9-4abd-adb7-07efe3b84c33 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:41:58.919609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab6825-47f9-4abd-adb7-07efe3b84c33 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:41:58.936565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:41:58.953684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6523d-171b-4c9b-b55d-0a304815461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:41:58.954546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6523d-171b-4c9b-b55d-0a304815461c None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:41:58.954825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6523d-171b-4c9b-b55d-0a304815461c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:41:58.955095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6523d-171b-4c9b-b55d-0a304815461c None admin] PATH_INFO: `/v3/projects/46185aeb22dd418fa143b0ec720e8231/users/ba9468feae5144c78f961dfe239cf73b/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:41:58.957049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6523d-171b-4c9b-b55d-0a304815461c None admin] RBAC: Authorizing `identity:create_grant(project_id=46185aeb22dd418fa143b0ec720e8231, user_id=ba9468feae5144c78f961dfe239cf73b, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:41:58.958507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6523d-171b-4c9b-b55d-0a304815461c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:41:58.971243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 594/1192] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:41:58 2022] PUT /identity/v3/projects/46185aeb22dd418fa143b0ec720e8231/users/ba9468feae5144c78f961dfe239cf73b/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:42:00.975864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69195d8c-cff7-4307-811e-6d49b7b30c10 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:00.976391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69195d8c-cff7-4307-811e-6d49b7b30c10 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:00.976391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69195d8c-cff7-4307-811e-6d49b7b30c10 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:00.977811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 599/1193] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:42:00.982955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b36fa-0cbd-416e-b320-e57ab9badc94 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:00.983357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b36fa-0cbd-416e-b320-e57ab9badc94 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:00.983513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b36fa-0cbd-416e-b320-e57ab9badc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:01.000497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5c7b36fa-0cbd-416e-b320-e57ab9badc94 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:42:01.001710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5c7b36fa-0cbd-416e-b320-e57ab9badc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:42:01.067571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 595/1194] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:42:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:42:01.091415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69195d8c-cff7-4307-811e-6d49b7b30c10 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:42:01.131578 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:42:01.192391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69195d8c-cff7-4307-811e-6d49b7b30c10 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:42:01.225063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1580bc-02c8-49e1-a924-41c0f2a4ec2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:42:01.225862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1580bc-02c8-49e1-a924-41c0f2a4ec2f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:01.226191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1580bc-02c8-49e1-a924-41c0f2a4ec2f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:01.226417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1580bc-02c8-49e1-a924-41c0f2a4ec2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:01.237027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1580bc-02c8-49e1-a924-41c0f2a4ec2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-cc1580bc-02c8-49e1-a924-41c0f2a4ec2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:42:01.244740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:42:01.245714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:42:01.246537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 600/1195] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:42:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:42:01.252008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b36fa-0cbd-416e-b320-e57ab9badc94 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:42:01.347163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b36fa-0cbd-416e-b320-e57ab9badc94 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:42:01.356513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:42:01.382143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d7114-214f-4243-ab45-bafffaf1467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:42:01.382802 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d7114-214f-4243-ab45-bafffaf1467c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:01.383084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d7114-214f-4243-ab45-bafffaf1467c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:01.383357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d7114-214f-4243-ab45-bafffaf1467c None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:01.384094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d7114-214f-4243-ab45-bafffaf1467c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:42:01.384397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d7114-214f-4243-ab45-bafffaf1467c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:42:01.385791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d7114-214f-4243-ab45-bafffaf1467c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:42:01.393678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 596/1196] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:42:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:42:01.399324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1580bc-02c8-49e1-a924-41c0f2a4ec2f None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:42:01.493743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:42:01.497802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1580bc-02c8-49e1-a924-41c0f2a4ec2f None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:42:01.531400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21186474-4ffa-42be-96c7-d8c36f1a0695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:42:01.532262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21186474-4ffa-42be-96c7-d8c36f1a0695 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:01.532544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21186474-4ffa-42be-96c7-d8c36f1a0695 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:01.532818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21186474-4ffa-42be-96c7-d8c36f1a0695 None admin] PATH_INFO: `/v3/users/ba9468feae5144c78f961dfe239cf73b` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:01.534291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21186474-4ffa-42be-96c7-d8c36f1a0695 None admin] RBAC: Authorizing `identity:get_user(user_id=ba9468feae5144c78f961dfe239cf73b)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:42:01.535727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21186474-4ffa-42be-96c7-d8c36f1a0695 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:42:01.537363 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 601/1197] 10.222.0.66 () {56 vars in 1177 bytes} [Fri Jul 1 19:42:01 2022] GET /identity/v3/users/ba9468feae5144c78f961dfe239cf73b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:42:01.543706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d7114-214f-4243-ab45-bafffaf1467c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:42:01.642547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d7114-214f-4243-ab45-bafffaf1467c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:42:01.675399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3014a-76e8-4d63-80b9-697b6c5708fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:42:01.676037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3014a-76e8-4d63-80b9-697b6c5708fc None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:01.676318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3014a-76e8-4d63-80b9-697b6c5708fc None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:01.676591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3014a-76e8-4d63-80b9-697b6c5708fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:01.714019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:42:01.742327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3014a-76e8-4d63-80b9-697b6c5708fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:42:01.743827 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3014a-76e8-4d63-80b9-697b6c5708fc None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:42:01.841534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 597/1198] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:42:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:42:01.847488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21186474-4ffa-42be-96c7-d8c36f1a0695 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:42:01.945671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21186474-4ffa-42be-96c7-d8c36f1a0695 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:42:01.978207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89294e-adea-4809-ac31-14ce112b1d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:42:01.979049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89294e-adea-4809-ac31-14ce112b1d75 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:01.979329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89294e-adea-4809-ac31-14ce112b1d75 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:01.979600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89294e-adea-4809-ac31-14ce112b1d75 None admin] PATH_INFO: `/v3/projects/46185aeb22dd418fa143b0ec720e8231` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:01.981456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89294e-adea-4809-ac31-14ce112b1d75 None admin] RBAC: Authorizing `identity:get_project(project_id=46185aeb22dd418fa143b0ec720e8231)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:42:01.982926 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89294e-adea-4809-ac31-14ce112b1d75 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:42:01.984604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 602/1199] 10.222.0.66 () {56 vars in 1186 bytes} [Fri Jul 1 19:42:01 2022] GET /identity/v3/projects/46185aeb22dd418fa143b0ec720e8231 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:42:01.990596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3014a-76e8-4d63-80b9-697b6c5708fc None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:42:02.063245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:42:02.098719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3014a-76e8-4d63-80b9-697b6c5708fc None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:42:02.131291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227fd81-4c1e-44f7-8f57-b7636b1eba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:42:02.131948 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227fd81-4c1e-44f7-8f57-b7636b1eba99 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:02.132263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227fd81-4c1e-44f7-8f57-b7636b1eba99 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:02.132568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227fd81-4c1e-44f7-8f57-b7636b1eba99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:02.133434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227fd81-4c1e-44f7-8f57-b7636b1eba99 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=46185aeb22dd418fa143b0ec720e8231, user.id=ba9468feae5144c78f961dfe239cf73b) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:42:02.133783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227fd81-4c1e-44f7-8f57-b7636b1eba99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:42:02.135238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227fd81-4c1e-44f7-8f57-b7636b1eba99 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:42:02.141062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 598/1200] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:42:01 2022] GET /identity/v3/role_assignments?user.id=ba9468feae5144c78f961dfe239cf73b&scope.project.id=46185aeb22dd418fa143b0ec720e8231&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:42:11.974166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=83 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:42:12.138515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=33 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:42:57.825128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67b013-18c7-4e52-9a73-b82c3e782656 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:57.825843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67b013-18c7-4e52-9a73-b82c3e782656 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:57.825843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67b013-18c7-4e52-9a73-b82c3e782656 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:57.827215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 603/1201] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:42:57.833110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610a329-6981-401d-9d83-d79388460b78 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:57.834036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610a329-6981-401d-9d83-d79388460b78 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:57.834515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610a329-6981-401d-9d83-d79388460b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:57.853450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c610a329-6981-401d-9d83-d79388460b78 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:42:57.856951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c610a329-6981-401d-9d83-d79388460b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:42:57.943087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 599/1202] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:42:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:42:57.975046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67b013-18c7-4e52-9a73-b82c3e782656 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:42:58.107618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67b013-18c7-4e52-9a73-b82c3e782656 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:42:58.152947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af374f99-598f-4e9d-9637-ca0d75b3df6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:42:58.154149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af374f99-598f-4e9d-9637-ca0d75b3df6d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:58.154472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af374f99-598f-4e9d-9637-ca0d75b3df6d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:58.154740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af374f99-598f-4e9d-9637-ca0d75b3df6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:58.167528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af374f99-598f-4e9d-9637-ca0d75b3df6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:42:58.169038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af374f99-598f-4e9d-9637-ca0d75b3df6d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-af374f99-598f-4e9d-9637-ca0d75b3df6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:42:58.193500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:42:58.194761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:42:58.195707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:42:58.195707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:42:58.195707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:42:58.195707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:42:58.195707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 604/1203] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:42:58.204049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610a329-6981-401d-9d83-d79388460b78 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:42:58.336810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610a329-6981-401d-9d83-d79388460b78 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:42:58.374652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:42:58.384331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae99f65-e6a0-4f66-a718-68d8654fc0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:42:58.385279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae99f65-e6a0-4f66-a718-68d8654fc0b2 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:58.385560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae99f65-e6a0-4f66-a718-68d8654fc0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:58.385867 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae99f65-e6a0-4f66-a718-68d8654fc0b2 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:58.386619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae99f65-e6a0-4f66-a718-68d8654fc0b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:42:58.386925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae99f65-e6a0-4f66-a718-68d8654fc0b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:42:58.388482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae99f65-e6a0-4f66-a718-68d8654fc0b2 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:42:58.403861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 600/1204] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:42:58.409843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af374f99-598f-4e9d-9637-ca0d75b3df6d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:42:58.491648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:42:58.558055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af374f99-598f-4e9d-9637-ca0d75b3df6d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:42:58.605628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e043803-76a6-471e-a6f2-bdc81de95723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:42:58.605628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e043803-76a6-471e-a6f2-bdc81de95723 None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:42:58.605628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e043803-76a6-471e-a6f2-bdc81de95723 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:42:58.606042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e043803-76a6-471e-a6f2-bdc81de95723 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:42:58.606681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e043803-76a6-471e-a6f2-bdc81de95723 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:42:58.608146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e043803-76a6-471e-a6f2-bdc81de95723 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:42:58.644034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 605/1205] 10.222.0.66 () {60 vars in 1132 bytes} [Fri Jul 1 19:42:58 2022] POST /identity/v3/users => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:43:00.996676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481de593-cfda-41c6-9237-eb3deeb9b2ee None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:00.997312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481de593-cfda-41c6-9237-eb3deeb9b2ee None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:00.997312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481de593-cfda-41c6-9237-eb3deeb9b2ee None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:00.998978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 601/1206] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:43:01.005808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a9f1b-fecc-452e-ad4a-f462b1e04476 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:01.006354 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a9f1b-fecc-452e-ad4a-f462b1e04476 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:01.006354 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a9f1b-fecc-452e-ad4a-f462b1e04476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:01.030425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-4f3a9f1b-fecc-452e-ad4a-f462b1e04476 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:43:01.031825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4f3a9f1b-fecc-452e-ad4a-f462b1e04476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:43:01.117343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 606/1207] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:43:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:43:01.144291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481de593-cfda-41c6-9237-eb3deeb9b2ee None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:01.270834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481de593-cfda-41c6-9237-eb3deeb9b2ee None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:01.315370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dc223-3fd8-4b54-9e82-2f695ea55f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:01.316169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dc223-3fd8-4b54-9e82-2f695ea55f15 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:01.316440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dc223-3fd8-4b54-9e82-2f695ea55f15 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:01.316703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dc223-3fd8-4b54-9e82-2f695ea55f15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:01.328798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66dc223-3fd8-4b54-9e82-2f695ea55f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-c66dc223-3fd8-4b54-9e82-2f695ea55f15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:43:01.338300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:01.339380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:01.345757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 602/1208] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:43:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:01.346511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a9f1b-fecc-452e-ad4a-f462b1e04476 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:01.390063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:01.469572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a9f1b-fecc-452e-ad4a-f462b1e04476 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:01.517724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b936c-4a31-4431-87e1-12943f9122b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:01.518832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b936c-4a31-4431-87e1-12943f9122b8 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:01.519261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b936c-4a31-4431-87e1-12943f9122b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:01.519550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b936c-4a31-4431-87e1-12943f9122b8 None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:01.520439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b936c-4a31-4431-87e1-12943f9122b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:01.520750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b936c-4a31-4431-87e1-12943f9122b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:01.522750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b936c-4a31-4431-87e1-12943f9122b8 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:01.537685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 607/1209] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:43:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:01.544886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dc223-3fd8-4b54-9e82-2f695ea55f15 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:01.611980 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:01.680910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dc223-3fd8-4b54-9e82-2f695ea55f15 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:01.720566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825b25c-dd9d-4ea8-ab48-8d013f18d50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:01.721471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825b25c-dd9d-4ea8-ab48-8d013f18d50f None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:01.721769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825b25c-dd9d-4ea8-ab48-8d013f18d50f None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:01.722104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825b25c-dd9d-4ea8-ab48-8d013f18d50f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:01.738656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825b25c-dd9d-4ea8-ab48-8d013f18d50f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:01.740191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825b25c-dd9d-4ea8-ab48-8d013f18d50f None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-6825b25c-dd9d-4ea8-ab48-8d013f18d50f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:01.762902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:01.763932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:01.764876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:01.764876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:01.764876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:01.764876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:01.764876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 603/1210] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:01.770016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b936c-4a31-4431-87e1-12943f9122b8 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:01.893713 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:01.897112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b936c-4a31-4431-87e1-12943f9122b8 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:01.941395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108fe1d-8557-4f78-8e23-374c7d2d47d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:01.941925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108fe1d-8557-4f78-8e23-374c7d2d47d6 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:01.942340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108fe1d-8557-4f78-8e23-374c7d2d47d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:01.942815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108fe1d-8557-4f78-8e23-374c7d2d47d6 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:01.943713 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108fe1d-8557-4f78-8e23-374c7d2d47d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:01.944122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108fe1d-8557-4f78-8e23-374c7d2d47d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:01.945740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108fe1d-8557-4f78-8e23-374c7d2d47d6 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:01.961253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 608/1211] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:01.966234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825b25c-dd9d-4ea8-ab48-8d013f18d50f None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:02.098918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825b25c-dd9d-4ea8-ab48-8d013f18d50f None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:02.101514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:02.147721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc5dc5-f49d-4177-86c3-8140a89c8ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:02.148838 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc5dc5-f49d-4177-86c3-8140a89c8ba1 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:02.149973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc5dc5-f49d-4177-86c3-8140a89c8ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:02.150196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc5dc5-f49d-4177-86c3-8140a89c8ba1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:02.162600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc5dc5-f49d-4177-86c3-8140a89c8ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:02.166246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc5dc5-f49d-4177-86c3-8140a89c8ba1 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-c2fc5dc5-f49d-4177-86c3-8140a89c8ba1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:43:02.192599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:02.194285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:02.195237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 604/1212] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:02.201892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108fe1d-8557-4f78-8e23-374c7d2d47d6 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:02.322538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108fe1d-8557-4f78-8e23-374c7d2d47d6 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:02.364260 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631af43-5e09-48cc-ac5c-244ee31fb4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:02.365282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631af43-5e09-48cc-ac5c-244ee31fb4b2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:02.365699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631af43-5e09-48cc-ac5c-244ee31fb4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:02.366010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631af43-5e09-48cc-ac5c-244ee31fb4b2 None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:02.367007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631af43-5e09-48cc-ac5c-244ee31fb4b2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:02.367368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631af43-5e09-48cc-ac5c-244ee31fb4b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:02.368974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631af43-5e09-48cc-ac5c-244ee31fb4b2 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:02.394242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 609/1213] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 1 19:43:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:02.396011 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc5dc5-f49d-4177-86c3-8140a89c8ba1 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:02.510845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc5dc5-f49d-4177-86c3-8140a89c8ba1 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:02.548794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966784f-c62e-422d-95d8-10c222a53c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:02.549750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966784f-c62e-422d-95d8-10c222a53c3c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:02.550186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966784f-c62e-422d-95d8-10c222a53c3c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:02.550804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966784f-c62e-422d-95d8-10c222a53c3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:02.587106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:02.634846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966784f-c62e-422d-95d8-10c222a53c3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:02.636542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966784f-c62e-422d-95d8-10c222a53c3c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:02.770921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 605/1214] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:43:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:43:02.776724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631af43-5e09-48cc-ac5c-244ee31fb4b2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:02.801331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:02.891852 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631af43-5e09-48cc-ac5c-244ee31fb4b2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:02.929108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb6588-f741-4fba-9189-9863833c4b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:02.930128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb6588-f741-4fba-9189-9863833c4b04 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:02.931036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb6588-f741-4fba-9189-9863833c4b04 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:02.931429 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb6588-f741-4fba-9189-9863833c4b04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:02.942358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb6588-f741-4fba-9189-9863833c4b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:02.943979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb6588-f741-4fba-9189-9863833c4b04 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-bdeb6588-f741-4fba-9189-9863833c4b04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:02.964052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:02.965201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:02.966439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:02.966439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:02.966439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:02.966439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:02.967336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 610/1215] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:02.973018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966784f-c62e-422d-95d8-10c222a53c3c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:03.040912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:03.083618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966784f-c62e-422d-95d8-10c222a53c3c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:03.123409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba44050-ec59-4d03-9fc1-1c45afbb8830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:03.124061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba44050-ec59-4d03-9fc1-1c45afbb8830 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:03.124339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba44050-ec59-4d03-9fc1-1c45afbb8830 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:03.124616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba44050-ec59-4d03-9fc1-1c45afbb8830 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:03.125338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba44050-ec59-4d03-9fc1-1c45afbb8830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:03.125666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba44050-ec59-4d03-9fc1-1c45afbb8830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:03.127232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba44050-ec59-4d03-9fc1-1c45afbb8830 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:03.141616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 606/1216] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:43:03.147652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb6588-f741-4fba-9189-9863833c4b04 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:03.232936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:03.268854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb6588-f741-4fba-9189-9863833c4b04 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:03.310193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21cd77-b50c-4640-b984-6c0078158903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:03.311113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21cd77-b50c-4640-b984-6c0078158903 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:03.311475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21cd77-b50c-4640-b984-6c0078158903 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:03.311750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21cd77-b50c-4640-b984-6c0078158903 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:03.320037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21cd77-b50c-4640-b984-6c0078158903 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:03.321851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21cd77-b50c-4640-b984-6c0078158903 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-3c21cd77-b50c-4640-b984-6c0078158903 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:43:03.339204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:03.340463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:03.341462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:43:03.341462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:43:03.341462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:43:03.341462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:43:03.341462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:43:03.341462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:43:03.341462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:03.341462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:03.341462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 611/1217] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:03.347213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba44050-ec59-4d03-9fc1-1c45afbb8830 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:03.466220 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba44050-ec59-4d03-9fc1-1c45afbb8830 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:03.490861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:03.504387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff76497-7cfd-4398-8ab0-e45c30367e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:03.505057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff76497-7cfd-4398-8ab0-e45c30367e55 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:03.505336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff76497-7cfd-4398-8ab0-e45c30367e55 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:03.505626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff76497-7cfd-4398-8ab0-e45c30367e55 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:03.506419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff76497-7cfd-4398-8ab0-e45c30367e55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:03.506723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff76497-7cfd-4398-8ab0-e45c30367e55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:03.508251 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff76497-7cfd-4398-8ab0-e45c30367e55 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:03.527329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 607/1218] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:43:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:03.528610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21cd77-b50c-4640-b984-6c0078158903 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:03.654467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21cd77-b50c-4640-b984-6c0078158903 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:03.685611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:03.691892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d48ab-426a-4651-95b4-5c72fed8d25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:03.692538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d48ab-426a-4651-95b4-5c72fed8d25b None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:03.692819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d48ab-426a-4651-95b4-5c72fed8d25b None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:03.693090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d48ab-426a-4651-95b4-5c72fed8d25b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:03.694158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d48ab-426a-4651-95b4-5c72fed8d25b None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=bba7d9205e694f2182d6091061d1219c) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:03.694471 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d48ab-426a-4651-95b4-5c72fed8d25b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:03.696057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d48ab-426a-4651-95b4-5c72fed8d25b None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:03.701906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 612/1219] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:43:03 2022] GET /identity/v3/role_assignments?user.id=bba7d9205e694f2182d6091061d1219c&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:05.867547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed2286-a0b5-4007-a5d5-281d7c6d3ab0 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:05.868203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed2286-a0b5-4007-a5d5-281d7c6d3ab0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:05.868203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed2286-a0b5-4007-a5d5-281d7c6d3ab0 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:05.869472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 608/1220] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:43:05.874787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd000a-9176-4d9e-b440-67b548a8838c None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:05.875227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd000a-9176-4d9e-b440-67b548a8838c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:05.875381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd000a-9176-4d9e-b440-67b548a8838c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:05.894088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-cddd000a-9176-4d9e-b440-67b548a8838c None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:43:05.895404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-cddd000a-9176-4d9e-b440-67b548a8838c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:43:05.974089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 613/1221] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:43:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:43:05.999768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed2286-a0b5-4007-a5d5-281d7c6d3ab0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:06.114740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed2286-a0b5-4007-a5d5-281d7c6d3ab0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:06.155420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94039fc-2511-4250-96f8-8e7587236519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:06.156138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94039fc-2511-4250-96f8-8e7587236519 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:06.156416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94039fc-2511-4250-96f8-8e7587236519 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:06.156688 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94039fc-2511-4250-96f8-8e7587236519 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:06.167997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94039fc-2511-4250-96f8-8e7587236519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-c94039fc-2511-4250-96f8-8e7587236519 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:43:06.175780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:06.177077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:06.178096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 609/1222] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:43:06.183252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd000a-9176-4d9e-b440-67b548a8838c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:06.289720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd000a-9176-4d9e-b440-67b548a8838c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:06.316958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:06.328993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf19a6d-31d6-42f2-bb81-68d7887ec93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:06.329714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf19a6d-31d6-42f2-bb81-68d7887ec93d None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:06.330015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf19a6d-31d6-42f2-bb81-68d7887ec93d None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:06.330281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf19a6d-31d6-42f2-bb81-68d7887ec93d None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:06.331025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf19a6d-31d6-42f2-bb81-68d7887ec93d None admin] RBAC: Adding query filter params (name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:06.331351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf19a6d-31d6-42f2-bb81-68d7887ec93d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:06.332717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf19a6d-31d6-42f2-bb81-68d7887ec93d None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:06.341037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 614/1223] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:43:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:06.346874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94039fc-2511-4250-96f8-8e7587236519 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:06.395374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:06.464643 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94039fc-2511-4250-96f8-8e7587236519 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:06.510154 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa0d7c-f818-4e44-a48f-50e1eda09824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:06.511078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa0d7c-f818-4e44-a48f-50e1eda09824 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:06.511417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa0d7c-f818-4e44-a48f-50e1eda09824 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:06.511749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa0d7c-f818-4e44-a48f-50e1eda09824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:06.523465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa0d7c-f818-4e44-a48f-50e1eda09824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:06.525684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa0d7c-f818-4e44-a48f-50e1eda09824 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-aafa0d7c-f818-4e44-a48f-50e1eda09824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:06.545562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:06.546589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:06.547680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:06.547680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:06.547680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:06.547680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:06.547680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 610/1224] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:06.553909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf19a6d-31d6-42f2-bb81-68d7887ec93d None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:06.662139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf19a6d-31d6-42f2-bb81-68d7887ec93d None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:06.701181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a610d0f-8d1e-4e1f-8759-d85c09254c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:06.702121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a610d0f-8d1e-4e1f-8759-d85c09254c58 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:06.712036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a610d0f-8d1e-4e1f-8759-d85c09254c58 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:06.712385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a610d0f-8d1e-4e1f-8759-d85c09254c58 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:06.713466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a610d0f-8d1e-4e1f-8759-d85c09254c58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:06.713805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a610d0f-8d1e-4e1f-8759-d85c09254c58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:06.715353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a610d0f-8d1e-4e1f-8759-d85c09254c58 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:06.727710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 615/1225] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:43:06.734426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa0d7c-f818-4e44-a48f-50e1eda09824 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:06.832762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:06.842512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa0d7c-f818-4e44-a48f-50e1eda09824 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:06.880410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220d18c-f516-47f2-8423-08a6ce382bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:06.881206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220d18c-f516-47f2-8423-08a6ce382bdd None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:06.881484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220d18c-f516-47f2-8423-08a6ce382bdd None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:06.881783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220d18c-f516-47f2-8423-08a6ce382bdd None admin] PATH_INFO: `/v3/users/placement` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:06.890093 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220d18c-f516-47f2-8423-08a6ce382bdd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:06.891541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220d18c-f516-47f2-8423-08a6ce382bdd None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-0220d18c-f516-47f2-8423-08a6ce382bdd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:43:06.907696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:06.908888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:06.910235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 611/1226] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:06.915160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a610d0f-8d1e-4e1f-8759-d85c09254c58 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:06.970467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:07.032325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a610d0f-8d1e-4e1f-8759-d85c09254c58 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:07.073036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147969b-bb41-476e-b4f1-31b8152a0a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:07.073788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147969b-bb41-476e-b4f1-31b8152a0a6c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:07.074112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147969b-bb41-476e-b4f1-31b8152a0a6c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:07.074492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147969b-bb41-476e-b4f1-31b8152a0a6c None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:07.075468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147969b-bb41-476e-b4f1-31b8152a0a6c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:07.075784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147969b-bb41-476e-b4f1-31b8152a0a6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:07.077473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147969b-bb41-476e-b4f1-31b8152a0a6c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:07.097750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 616/1227] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 1 19:43:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:07.104362 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220d18c-f516-47f2-8423-08a6ce382bdd None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:07.226479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220d18c-f516-47f2-8423-08a6ce382bdd None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:07.265248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a12bf-c71e-4e0a-acf4-f9803e7e97db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:07.266153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a12bf-c71e-4e0a-acf4-f9803e7e97db None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:07.266458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a12bf-c71e-4e0a-acf4-f9803e7e97db None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:07.266809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a12bf-c71e-4e0a-acf4-f9803e7e97db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:07.279318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a12bf-c71e-4e0a-acf4-f9803e7e97db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:07.280898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a12bf-c71e-4e0a-acf4-f9803e7e97db None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-699a12bf-c71e-4e0a-acf4-f9803e7e97db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:07.300719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:07.301856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:07.302954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:07.302954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:07.302954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:07.302954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:07.302954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 612/1228] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:07.310604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147969b-bb41-476e-b4f1-31b8152a0a6c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:07.420752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147969b-bb41-476e-b4f1-31b8152a0a6c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:07.423043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:07.459334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4e736-0765-44f2-8618-951366b44c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:07.459994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4e736-0765-44f2-8618-951366b44c50 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:07.460274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4e736-0765-44f2-8618-951366b44c50 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:07.460548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4e736-0765-44f2-8618-951366b44c50 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:07.461327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4e736-0765-44f2-8618-951366b44c50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:07.461862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4e736-0765-44f2-8618-951366b44c50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:07.463266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4e736-0765-44f2-8618-951366b44c50 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:07.475691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 617/1229] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:43:07.482587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a12bf-c71e-4e0a-acf4-f9803e7e97db None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:07.494326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:07.598725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a12bf-c71e-4e0a-acf4-f9803e7e97db None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:07.635468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190fc3a-1980-4e89-986b-4daf3af0f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:07.636264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190fc3a-1980-4e89-986b-4daf3af0f36e None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:07.636548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190fc3a-1980-4e89-986b-4daf3af0f36e None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:07.636815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190fc3a-1980-4e89-986b-4daf3af0f36e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:07.644914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190fc3a-1980-4e89-986b-4daf3af0f36e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:07.646399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190fc3a-1980-4e89-986b-4daf3af0f36e None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-c190fc3a-1980-4e89-986b-4daf3af0f36e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:43:07.662852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:07.663971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:07.666204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:43:07.666204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:43:07.666204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:43:07.666204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:43:07.666204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:43:07.666204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:43:07.666204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:07.666204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:07.666204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 613/1230] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:43:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:07.670858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4e736-0765-44f2-8618-951366b44c50 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:07.779248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4e736-0765-44f2-8618-951366b44c50 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:07.817564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef5e26-e65b-4534-bac4-db7ee38b9ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:07.818298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5e26-e65b-4534-bac4-db7ee38b9ab7 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:07.818654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5e26-e65b-4534-bac4-db7ee38b9ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:07.818955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5e26-e65b-4534-bac4-db7ee38b9ab7 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:07.819946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef5e26-e65b-4534-bac4-db7ee38b9ab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:07.820262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef5e26-e65b-4534-bac4-db7ee38b9ab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:07.822569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef5e26-e65b-4534-bac4-db7ee38b9ab7 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:07.835775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 618/1231] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:43:07.843509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190fc3a-1980-4e89-986b-4daf3af0f36e None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:07.918905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:07.959723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190fc3a-1980-4e89-986b-4daf3af0f36e None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:08.000756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33cb1b-74d7-4158-b19b-2af4df4ed8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:08.001664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33cb1b-74d7-4158-b19b-2af4df4ed8b4 None admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:08.001944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33cb1b-74d7-4158-b19b-2af4df4ed8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:08.002301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33cb1b-74d7-4158-b19b-2af4df4ed8b4 None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/bba7d9205e694f2182d6091061d1219c/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:08.018813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33cb1b-74d7-4158-b19b-2af4df4ed8b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=bba7d9205e694f2182d6091061d1219c, role_id=0db2f13c4dbe42a4a0d8ffa0bb5633fd)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:08.020490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33cb1b-74d7-4158-b19b-2af4df4ed8b4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:08.034175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 614/1232] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:43:07 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/bba7d9205e694f2182d6091061d1219c/roles/0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:43:10.202499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8b663-8bcd-4859-bd9d-62379a6103fe None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:10.203109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8b663-8bcd-4859-bd9d-62379a6103fe None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:10.203109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8b663-8bcd-4859-bd9d-62379a6103fe None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:10.204482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 619/1233] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:43:10.209887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e838c-8c6f-41a8-b7e3-2cb6baba8269 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:10.210396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e838c-8c6f-41a8-b7e3-2cb6baba8269 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:10.210511 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e838c-8c6f-41a8-b7e3-2cb6baba8269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:10.228190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-2b3e838c-8c6f-41a8-b7e3-2cb6baba8269 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:43:10.229368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2b3e838c-8c6f-41a8-b7e3-2cb6baba8269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:43:10.303909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 615/1234] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:43:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:43:10.330289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8b663-8bcd-4859-bd9d-62379a6103fe None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:10.358623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:10.450750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8b663-8bcd-4859-bd9d-62379a6103fe None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:10.488167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d62b6-d159-41c3-a3c7-717a9407554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:10.488891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d62b6-d159-41c3-a3c7-717a9407554f None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:10.489174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d62b6-d159-41c3-a3c7-717a9407554f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:10.489465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d62b6-d159-41c3-a3c7-717a9407554f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:10.501306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d62b6-d159-41c3-a3c7-717a9407554f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-083d62b6-d159-41c3-a3c7-717a9407554f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:43:10.509154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:10.510333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:10.511279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 620/1235] 10.222.0.66 () {56 vars in 1102 bytes} [Fri Jul 1 19:43:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:43:10.516279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e838c-8c6f-41a8-b7e3-2cb6baba8269 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:10.567706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:10.624503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e838c-8c6f-41a8-b7e3-2cb6baba8269 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:10.661550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b636636-5067-4c92-9bff-25ab695b7b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:10.662354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b636636-5067-4c92-9bff-25ab695b7b40 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:10.662636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b636636-5067-4c92-9bff-25ab695b7b40 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:10.662909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b636636-5067-4c92-9bff-25ab695b7b40 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:10.663805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b636636-5067-4c92-9bff-25ab695b7b40 None admin] RBAC: Adding query filter params (name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:10.664650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b636636-5067-4c92-9bff-25ab695b7b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:10.666382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b636636-5067-4c92-9bff-25ab695b7b40 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:10.676405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 616/1236] 10.222.0.66 () {56 vars in 1103 bytes} [Fri Jul 1 19:43:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:10.683086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d62b6-d159-41c3-a3c7-717a9407554f None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:10.777198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:10.796247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d62b6-d159-41c3-a3c7-717a9407554f None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:10.835575 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d551d2d-7d44-415f-a673-d47f8e78cfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:10.837051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d551d2d-7d44-415f-a673-d47f8e78cfd4 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:10.837328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d551d2d-7d44-415f-a673-d47f8e78cfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:10.837767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d551d2d-7d44-415f-a673-d47f8e78cfd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:10.853431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d551d2d-7d44-415f-a673-d47f8e78cfd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:10.855666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d551d2d-7d44-415f-a673-d47f8e78cfd4 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-4d551d2d-7d44-415f-a673-d47f8e78cfd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:10.875496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:10.876599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:10.877871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:10.877871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:10.877871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:10.877871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:10.877871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 621/1237] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:10.882607 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b636636-5067-4c92-9bff-25ab695b7b40 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:10.990586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b636636-5067-4c92-9bff-25ab695b7b40 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:11.000720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:11.028579 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b0d66-17c2-4a75-ba07-cf79b6ad8e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:11.029272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b0d66-17c2-4a75-ba07-cf79b6ad8e62 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:11.029606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b0d66-17c2-4a75-ba07-cf79b6ad8e62 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:11.029907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b0d66-17c2-4a75-ba07-cf79b6ad8e62 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:11.030727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b0d66-17c2-4a75-ba07-cf79b6ad8e62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:11.031090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b0d66-17c2-4a75-ba07-cf79b6ad8e62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:11.032779 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b0d66-17c2-4a75-ba07-cf79b6ad8e62 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:11.045322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 617/1238] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:43:11.051080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d551d2d-7d44-415f-a673-d47f8e78cfd4 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:11.168134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d551d2d-7d44-415f-a673-d47f8e78cfd4 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:11.211934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54b710-9622-425e-a1e2-006b41003ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:11.212804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54b710-9622-425e-a1e2-006b41003ffc None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:11.213098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54b710-9622-425e-a1e2-006b41003ffc None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:11.213662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54b710-9622-425e-a1e2-006b41003ffc None admin] PATH_INFO: `/v3/users/placement` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:11.217237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:11.224665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54b710-9622-425e-a1e2-006b41003ffc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:11.227373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54b710-9622-425e-a1e2-006b41003ffc None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-fa54b710-9622-425e-a1e2-006b41003ffc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:43:11.246365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:11.247650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:11.248818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 622/1239] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:11.253987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b0d66-17c2-4a75-ba07-cf79b6ad8e62 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:11.366031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b0d66-17c2-4a75-ba07-cf79b6ad8e62 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:11.402110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce745f-6132-492f-beb2-1a1c11626d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:11.402778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce745f-6132-492f-beb2-1a1c11626d76 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:11.403075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce745f-6132-492f-beb2-1a1c11626d76 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:11.403368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce745f-6132-492f-beb2-1a1c11626d76 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:11.404150 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce745f-6132-492f-beb2-1a1c11626d76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:11.404449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce745f-6132-492f-beb2-1a1c11626d76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:11.405936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce745f-6132-492f-beb2-1a1c11626d76 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:11.423580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 618/1240] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 1 19:43:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:11.429418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54b710-9622-425e-a1e2-006b41003ffc None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:11.540382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54b710-9622-425e-a1e2-006b41003ffc None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:11.578261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798ca95-6dcf-4b33-9d2b-25c710f1a10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:11.578903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798ca95-6dcf-4b33-9d2b-25c710f1a10c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:11.579182 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798ca95-6dcf-4b33-9d2b-25c710f1a10c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:11.579455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798ca95-6dcf-4b33-9d2b-25c710f1a10c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:11.644843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:11.653303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798ca95-6dcf-4b33-9d2b-25c710f1a10c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:11.655664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798ca95-6dcf-4b33-9d2b-25c710f1a10c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:11.782540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 623/1241] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:43:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 19:43:11.788962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce745f-6132-492f-beb2-1a1c11626d76 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:11.819503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:11.903322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce745f-6132-492f-beb2-1a1c11626d76 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:11.942090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386dc60-e72b-4e9b-85da-8c7f9ee54066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:11.942904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386dc60-e72b-4e9b-85da-8c7f9ee54066 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:11.943170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386dc60-e72b-4e9b-85da-8c7f9ee54066 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:11.943445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386dc60-e72b-4e9b-85da-8c7f9ee54066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:11.954292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386dc60-e72b-4e9b-85da-8c7f9ee54066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:11.955872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386dc60-e72b-4e9b-85da-8c7f9ee54066 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-e386dc60-e72b-4e9b-85da-8c7f9ee54066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:11.974542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:11.975522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:11.977012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:11.977012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:11.977012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:11.977012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:11.977012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 619/1242] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:11.982250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798ca95-6dcf-4b33-9d2b-25c710f1a10c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:12.082438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:12.089052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798ca95-6dcf-4b33-9d2b-25c710f1a10c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:12.127254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7912b-d7d0-48f3-b9fe-d49a6e059713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:12.127910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7912b-d7d0-48f3-b9fe-d49a6e059713 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:12.128189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7912b-d7d0-48f3-b9fe-d49a6e059713 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:12.128464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7912b-d7d0-48f3-b9fe-d49a6e059713 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:12.129179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7912b-d7d0-48f3-b9fe-d49a6e059713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:12.129480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7912b-d7d0-48f3-b9fe-d49a6e059713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:12.131101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7912b-d7d0-48f3-b9fe-d49a6e059713 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:12.143299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 624/1243] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:43:12.149471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386dc60-e72b-4e9b-85da-8c7f9ee54066 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:12.255053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:12.280586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386dc60-e72b-4e9b-85da-8c7f9ee54066 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:12.318030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85c8b3-93be-4480-bb46-0ffe9582b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:12.318854 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85c8b3-93be-4480-bb46-0ffe9582b01b None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:12.319097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85c8b3-93be-4480-bb46-0ffe9582b01b None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:12.319399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85c8b3-93be-4480-bb46-0ffe9582b01b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:12.327077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85c8b3-93be-4480-bb46-0ffe9582b01b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:12.328509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85c8b3-93be-4480-bb46-0ffe9582b01b None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-9c85c8b3-93be-4480-bb46-0ffe9582b01b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:43:12.345949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:12.347061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:12.353651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:43:12.353651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:43:12.353651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:43:12.353651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:43:12.353651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:43:12.353651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:43:12.353651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:12.353651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:12.353651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 620/1244] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:12.354239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7912b-d7d0-48f3-b9fe-d49a6e059713 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:12.469436 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7912b-d7d0-48f3-b9fe-d49a6e059713 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:12.506395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182ec6b-af5c-48f9-ae64-3fd98fc68aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:12.507205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182ec6b-af5c-48f9-ae64-3fd98fc68aa0 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:12.507582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182ec6b-af5c-48f9-ae64-3fd98fc68aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:12.508415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182ec6b-af5c-48f9-ae64-3fd98fc68aa0 None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:12.509246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182ec6b-af5c-48f9-ae64-3fd98fc68aa0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:12.509671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182ec6b-af5c-48f9-ae64-3fd98fc68aa0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:12.511480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182ec6b-af5c-48f9-ae64-3fd98fc68aa0 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:12.525425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 625/1245] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:43:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:12.531918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85c8b3-93be-4480-bb46-0ffe9582b01b None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:12.651518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85c8b3-93be-4480-bb46-0ffe9582b01b None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:12.694921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb58089-c746-4d1b-9dd7-132ba36694d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:12.695883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb58089-c746-4d1b-9dd7-132ba36694d0 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:12.695883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb58089-c746-4d1b-9dd7-132ba36694d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:12.696158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb58089-c746-4d1b-9dd7-132ba36694d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:12.697139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb58089-c746-4d1b-9dd7-132ba36694d0 None admin] RBAC: Adding query filter params (role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=bba7d9205e694f2182d6091061d1219c) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:12.697673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb58089-c746-4d1b-9dd7-132ba36694d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:12.699539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb58089-c746-4d1b-9dd7-132ba36694d0 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:12.707021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 621/1246] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:43:12 2022] GET /identity/v3/role_assignments?user.id=bba7d9205e694f2182d6091061d1219c&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=0db2f13c4dbe42a4a0d8ffa0bb5633fd => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:43:14.861006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c12eb6-64b7-4fb1-a858-3f1a2955477d None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:14.861601 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c12eb6-64b7-4fb1-a858-3f1a2955477d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:14.861601 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c12eb6-64b7-4fb1-a858-3f1a2955477d None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:14.862967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 626/1247] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:43:14.869692 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9070c-2b80-4c55-a879-c206f64265fa None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:14.870155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9070c-2b80-4c55-a879-c206f64265fa None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:14.870278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9070c-2b80-4c55-a879-c206f64265fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:14.876239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:14.891445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-51b9070c-2b80-4c55-a879-c206f64265fa None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:43:14.892779 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-51b9070c-2b80-4c55-a879-c206f64265fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:43:14.970473 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 622/1248] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:43:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:43:14.995357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c12eb6-64b7-4fb1-a858-3f1a2955477d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:15.016433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:15.113311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c12eb6-64b7-4fb1-a858-3f1a2955477d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:15.153774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096e292-534c-40b7-8f73-b3385e353af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:15.154738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096e292-534c-40b7-8f73-b3385e353af2 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:15.155103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096e292-534c-40b7-8f73-b3385e353af2 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:15.155420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096e292-534c-40b7-8f73-b3385e353af2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:15.167287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096e292-534c-40b7-8f73-b3385e353af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-f096e292-534c-40b7-8f73-b3385e353af2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:43:15.175332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:15.176528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:15.177560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 627/1249] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:43:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:15.183974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9070c-2b80-4c55-a879-c206f64265fa None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:15.307173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9070c-2b80-4c55-a879-c206f64265fa None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:15.342027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:15.349506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608fe13-d4cd-4b62-b0c8-6bccbf8bd0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:15.350317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608fe13-d4cd-4b62-b0c8-6bccbf8bd0b3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:15.350718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608fe13-d4cd-4b62-b0c8-6bccbf8bd0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:15.351039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608fe13-d4cd-4b62-b0c8-6bccbf8bd0b3 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:15.352428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608fe13-d4cd-4b62-b0c8-6bccbf8bd0b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:15.352735 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608fe13-d4cd-4b62-b0c8-6bccbf8bd0b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:15.354143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608fe13-d4cd-4b62-b0c8-6bccbf8bd0b3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:15.363219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 623/1250] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:43:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:15.369855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096e292-534c-40b7-8f73-b3385e353af2 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:15.464224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:15.489633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096e292-534c-40b7-8f73-b3385e353af2 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:15.527718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ad428-0b26-41f7-9e38-180d6590e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:15.529019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ad428-0b26-41f7-9e38-180d6590e272 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:15.529019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ad428-0b26-41f7-9e38-180d6590e272 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:15.529195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ad428-0b26-41f7-9e38-180d6590e272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:15.541917 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ad428-0b26-41f7-9e38-180d6590e272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:15.543510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ad428-0b26-41f7-9e38-180d6590e272 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-b81ad428-0b26-41f7-9e38-180d6590e272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:15.562441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:15.563420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:15.564890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:15.564890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:15.564890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:15.564890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:15.564890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 628/1251] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:15.569920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608fe13-d4cd-4b62-b0c8-6bccbf8bd0b3 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:15.682572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608fe13-d4cd-4b62-b0c8-6bccbf8bd0b3 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:15.718624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a2e62-945f-4163-aa46-7f34a170e26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:15.719555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a2e62-945f-4163-aa46-7f34a170e26c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:15.719839 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a2e62-945f-4163-aa46-7f34a170e26c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:15.720108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a2e62-945f-4163-aa46-7f34a170e26c None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:15.720814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a2e62-945f-4163-aa46-7f34a170e26c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:15.721111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a2e62-945f-4163-aa46-7f34a170e26c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:15.722503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a2e62-945f-4163-aa46-7f34a170e26c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:15.736363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 624/1252] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:15.743147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ad428-0b26-41f7-9e38-180d6590e272 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:15.859572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ad428-0b26-41f7-9e38-180d6590e272 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:15.887232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:15.895389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f096948-76e0-4bc0-be4e-d14c8d36a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:15.896187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f096948-76e0-4bc0-be4e-d14c8d36a97a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:15.896465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f096948-76e0-4bc0-be4e-d14c8d36a97a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:15.896756 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f096948-76e0-4bc0-be4e-d14c8d36a97a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:15.904618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f096948-76e0-4bc0-be4e-d14c8d36a97a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:15.906079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f096948-76e0-4bc0-be4e-d14c8d36a97a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-9f096948-76e0-4bc0-be4e-d14c8d36a97a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:43:15.922011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:15.923054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:15.924458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 629/1253] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:15.928946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a2e62-945f-4163-aa46-7f34a170e26c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:15.989797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:16.040606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a2e62-945f-4163-aa46-7f34a170e26c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:16.078017 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c409af-b412-437a-a05f-b9269ac55e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:16.078675 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c409af-b412-437a-a05f-b9269ac55e89 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:16.078950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c409af-b412-437a-a05f-b9269ac55e89 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:16.079218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c409af-b412-437a-a05f-b9269ac55e89 None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:16.080049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c409af-b412-437a-a05f-b9269ac55e89 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:16.080348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c409af-b412-437a-a05f-b9269ac55e89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:16.081787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c409af-b412-437a-a05f-b9269ac55e89 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:16.100901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 625/1254] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 1 19:43:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:16.108742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f096948-76e0-4bc0-be4e-d14c8d36a97a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:16.228168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f096948-76e0-4bc0-be4e-d14c8d36a97a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:16.268938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636166e-b701-40e2-8069-2190064d2732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:16.269572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636166e-b701-40e2-8069-2190064d2732 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:16.269960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636166e-b701-40e2-8069-2190064d2732 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:16.270237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636166e-b701-40e2-8069-2190064d2732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:16.334967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:16.349939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636166e-b701-40e2-8069-2190064d2732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:16.351948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636166e-b701-40e2-8069-2190064d2732 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:16.458822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 630/1255] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:43:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 19:43:16.465697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c409af-b412-437a-a05f-b9269ac55e89 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:16.582496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c409af-b412-437a-a05f-b9269ac55e89 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:16.597111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:16.620103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5819e-8c71-4884-a245-fb9e0f7278cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:16.620895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5819e-8c71-4884-a245-fb9e0f7278cb None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:16.621175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5819e-8c71-4884-a245-fb9e0f7278cb None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:16.621451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5819e-8c71-4884-a245-fb9e0f7278cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:16.631546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5819e-8c71-4884-a245-fb9e0f7278cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:16.633566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5819e-8c71-4884-a245-fb9e0f7278cb None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-f3a5819e-8c71-4884-a245-fb9e0f7278cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:16.652206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:16.653171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:16.659417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:16.659417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:16.659417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:16.659417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:16.659417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 626/1256] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:16.659708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636166e-b701-40e2-8069-2190064d2732 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:16.752432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:16.768869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636166e-b701-40e2-8069-2190064d2732 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:16.808031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1242c-f518-426b-9f74-ec7a27862b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:16.808698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1242c-f518-426b-9f74-ec7a27862b7a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:16.808992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1242c-f518-426b-9f74-ec7a27862b7a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:16.809259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1242c-f518-426b-9f74-ec7a27862b7a None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:16.810000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1242c-f518-426b-9f74-ec7a27862b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:16.810300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1242c-f518-426b-9f74-ec7a27862b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:16.814759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1242c-f518-426b-9f74-ec7a27862b7a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:16.831916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 631/1257] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:16.837765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5819e-8c71-4884-a245-fb9e0f7278cb None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:16.953906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5819e-8c71-4884-a245-fb9e0f7278cb None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:16.995782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a84641-065c-4f45-97f2-10a55ab37d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:16.996617 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84641-065c-4f45-97f2-10a55ab37d04 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:16.996901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84641-065c-4f45-97f2-10a55ab37d04 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:16.997326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84641-065c-4f45-97f2-10a55ab37d04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:17.005467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a84641-065c-4f45-97f2-10a55ab37d04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:17.007078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a84641-065c-4f45-97f2-10a55ab37d04 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-51a84641-065c-4f45-97f2-10a55ab37d04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:43:17.023957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:17.025460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:17.026580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:43:17.026580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:43:17.026580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:43:17.026580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:43:17.026580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:43:17.026580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:43:17.026580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:17.026580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:17.026580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 627/1258] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:17.031693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1242c-f518-426b-9f74-ec7a27862b7a None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:17.138606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1242c-f518-426b-9f74-ec7a27862b7a None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:17.181479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cb201-ba43-4f81-a479-8c549b186a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:17.182328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cb201-ba43-4f81-a479-8c549b186a0a None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:17.182474 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cb201-ba43-4f81-a479-8c549b186a0a None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:17.182824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cb201-ba43-4f81-a479-8c549b186a0a None admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:17.183665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cb201-ba43-4f81-a479-8c549b186a0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:17.184033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cb201-ba43-4f81-a479-8c549b186a0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:17.185928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cb201-ba43-4f81-a479-8c549b186a0a None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:17.200673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 632/1259] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:43:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:17.207924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a84641-065c-4f45-97f2-10a55ab37d04 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:17.246197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:17.328537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a84641-065c-4f45-97f2-10a55ab37d04 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:17.366216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110e7f5-c7b8-43cc-92b2-06c5ebc1d977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:17.366919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110e7f5-c7b8-43cc-92b2-06c5ebc1d977 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:17.367631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110e7f5-c7b8-43cc-92b2-06c5ebc1d977 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:17.367631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110e7f5-c7b8-43cc-92b2-06c5ebc1d977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:17.368415 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110e7f5-c7b8-43cc-92b2-06c5ebc1d977 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=bba7d9205e694f2182d6091061d1219c) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:17.368723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110e7f5-c7b8-43cc-92b2-06c5ebc1d977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:17.370212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110e7f5-c7b8-43cc-92b2-06c5ebc1d977 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:17.376447 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 628/1260] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:43:17 2022] GET /identity/v3/role_assignments?user.id=bba7d9205e694f2182d6091061d1219c&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:19.395108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462ddfd-ad77-497a-9b9f-2788ac813d2d None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:19.395659 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462ddfd-ad77-497a-9b9f-2788ac813d2d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:19.395659 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462ddfd-ad77-497a-9b9f-2788ac813d2d None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:19.396997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 633/1261] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:43:19.402235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0078b-0575-4f1e-9560-3d7df80c0979 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:19.402642 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0078b-0575-4f1e-9560-3d7df80c0979 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:19.402783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0078b-0575-4f1e-9560-3d7df80c0979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:19.420168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-cbe0078b-0575-4f1e-9560-3d7df80c0979 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:43:19.421336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-cbe0078b-0575-4f1e-9560-3d7df80c0979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:43:19.485760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 629/1262] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:43:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:43:19.508494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462ddfd-ad77-497a-9b9f-2788ac813d2d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:19.519125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:19.608308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462ddfd-ad77-497a-9b9f-2788ac813d2d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:19.640717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b39fe6-6c85-4b0d-ac62-78f1dba7f1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:19.641490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b39fe6-6c85-4b0d-ac62-78f1dba7f1d0 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:19.641792 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b39fe6-6c85-4b0d-ac62-78f1dba7f1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:19.642060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b39fe6-6c85-4b0d-ac62-78f1dba7f1d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:19.651848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b39fe6-6c85-4b0d-ac62-78f1dba7f1d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-d1b39fe6-6c85-4b0d-ac62-78f1dba7f1d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:43:19.659612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:19.660725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:19.661795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 634/1263] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:43:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:19.668086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0078b-0575-4f1e-9560-3d7df80c0979 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:19.686380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:19.764700 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0078b-0575-4f1e-9560-3d7df80c0979 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:19.797695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265645eb-98aa-41b3-9c89-f07de8dca0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:19.798347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265645eb-98aa-41b3-9c89-f07de8dca0e4 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:19.798630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265645eb-98aa-41b3-9c89-f07de8dca0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:19.798905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265645eb-98aa-41b3-9c89-f07de8dca0e4 None admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:19.799648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265645eb-98aa-41b3-9c89-f07de8dca0e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:19.799954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265645eb-98aa-41b3-9c89-f07de8dca0e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:19.801322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265645eb-98aa-41b3-9c89-f07de8dca0e4 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:19.809696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 630/1264] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:43:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:19.815510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b39fe6-6c85-4b0d-ac62-78f1dba7f1d0 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:19.893680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:19.931252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b39fe6-6c85-4b0d-ac62-78f1dba7f1d0 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:19.963991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d22cfc-18cb-4117-92ae-45e9364a1a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:19.964768 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d22cfc-18cb-4117-92ae-45e9364a1a56 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:19.965047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d22cfc-18cb-4117-92ae-45e9364a1a56 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:19.965319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d22cfc-18cb-4117-92ae-45e9364a1a56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:19.975577 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d22cfc-18cb-4117-92ae-45e9364a1a56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:19.977711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d22cfc-18cb-4117-92ae-45e9364a1a56 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-c2d22cfc-18cb-4117-92ae-45e9364a1a56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:19.994643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:19.995632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:19.996507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:19.996507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:19.996507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:19.996507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:19.996507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 635/1265] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:20.001734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265645eb-98aa-41b3-9c89-f07de8dca0e4 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:20.083547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:20.103989 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265645eb-98aa-41b3-9c89-f07de8dca0e4 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:20.137536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9b910-0bf2-483f-8730-5159509cc7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:20.138223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9b910-0bf2-483f-8730-5159509cc7f3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:20.138506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9b910-0bf2-483f-8730-5159509cc7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:20.138782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9b910-0bf2-483f-8730-5159509cc7f3 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:20.139497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9b910-0bf2-483f-8730-5159509cc7f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:20.139801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9b910-0bf2-483f-8730-5159509cc7f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:20.141170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9b910-0bf2-483f-8730-5159509cc7f3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:20.153561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 631/1266] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:20.159751 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d22cfc-18cb-4117-92ae-45e9364a1a56 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:20.300800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d22cfc-18cb-4117-92ae-45e9364a1a56 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:20.327728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:20.343152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744818bb-463c-4997-a3d5-503f0f0bf914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:20.343960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744818bb-463c-4997-a3d5-503f0f0bf914 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:20.344240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744818bb-463c-4997-a3d5-503f0f0bf914 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:20.344510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744818bb-463c-4997-a3d5-503f0f0bf914 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:20.354851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744818bb-463c-4997-a3d5-503f0f0bf914 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:20.356348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744818bb-463c-4997-a3d5-503f0f0bf914 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-744818bb-463c-4997-a3d5-503f0f0bf914 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:43:20.375985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:20.377198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:20.378557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 636/1267] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:20.383125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9b910-0bf2-483f-8730-5159509cc7f3 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:20.494323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9b910-0bf2-483f-8730-5159509cc7f3 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:20.533925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7931c41-e12e-4ef0-a756-58a4b07c9d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:20.534587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7931c41-e12e-4ef0-a756-58a4b07c9d3c None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:20.534868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7931c41-e12e-4ef0-a756-58a4b07c9d3c None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:20.535138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7931c41-e12e-4ef0-a756-58a4b07c9d3c None admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:20.535997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7931c41-e12e-4ef0-a756-58a4b07c9d3c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:20.536300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7931c41-e12e-4ef0-a756-58a4b07c9d3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:20.537758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7931c41-e12e-4ef0-a756-58a4b07c9d3c None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:20.550235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:20.560868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 632/1268] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 1 19:43:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:20.566562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744818bb-463c-4997-a3d5-503f0f0bf914 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:20.695562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744818bb-463c-4997-a3d5-503f0f0bf914 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:20.729528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c09436-7bb4-47e3-b55c-4600137efeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:20.730331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c09436-7bb4-47e3-b55c-4600137efeeb None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:20.730611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c09436-7bb4-47e3-b55c-4600137efeeb None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:20.730885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c09436-7bb4-47e3-b55c-4600137efeeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:20.741056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c09436-7bb4-47e3-b55c-4600137efeeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:20.742811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c09436-7bb4-47e3-b55c-4600137efeeb None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-49c09436-7bb4-47e3-b55c-4600137efeeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:20.759447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:20.760498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:20.761651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:20.761651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:20.761651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:20.761651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:20.761651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 637/1269] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:20.766416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7931c41-e12e-4ef0-a756-58a4b07c9d3c None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:20.865034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7931c41-e12e-4ef0-a756-58a4b07c9d3c None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:20.899507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c5504-00af-46ed-bbc2-ae04ce46c259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:20.900274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c5504-00af-46ed-bbc2-ae04ce46c259 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:20.900557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c5504-00af-46ed-bbc2-ae04ce46c259 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:20.900834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c5504-00af-46ed-bbc2-ae04ce46c259 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:20.901549 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c5504-00af-46ed-bbc2-ae04ce46c259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:20.901898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c5504-00af-46ed-bbc2-ae04ce46c259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:20.903274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c5504-00af-46ed-bbc2-ae04ce46c259 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:20.916381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 633/1270] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:20.922119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c09436-7bb4-47e3-b55c-4600137efeeb None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:20.954513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:21.025224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c09436-7bb4-47e3-b55c-4600137efeeb None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:21.065720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42dac6-004f-40dc-8251-6ac7a346b2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:21.065720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42dac6-004f-40dc-8251-6ac7a346b2eb None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:21.065720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42dac6-004f-40dc-8251-6ac7a346b2eb None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:21.065720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42dac6-004f-40dc-8251-6ac7a346b2eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:21.073547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42dac6-004f-40dc-8251-6ac7a346b2eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:21.075200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42dac6-004f-40dc-8251-6ac7a346b2eb None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-ab42dac6-004f-40dc-8251-6ac7a346b2eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:43:21.090391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:21.091410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:21.092315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:43:21.092315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:43:21.092315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:43:21.092315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:43:21.092315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:43:21.092315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:43:21.092315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:21.092315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:21.092315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 638/1271] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:43:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:21.097520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c5504-00af-46ed-bbc2-ae04ce46c259 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:21.159645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:21.207126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c5504-00af-46ed-bbc2-ae04ce46c259 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:21.243321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7ef0d-6012-4b2b-8f98-d8645908b561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:21.243978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ef0d-6012-4b2b-8f98-d8645908b561 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:21.244259 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ef0d-6012-4b2b-8f98-d8645908b561 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:21.244736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ef0d-6012-4b2b-8f98-d8645908b561 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:21.245524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7ef0d-6012-4b2b-8f98-d8645908b561 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:21.245865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7ef0d-6012-4b2b-8f98-d8645908b561 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:21.247290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7ef0d-6012-4b2b-8f98-d8645908b561 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:21.259750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 634/1272] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:43:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:43:21.265858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42dac6-004f-40dc-8251-6ac7a346b2eb None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:21.354771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:21.363401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42dac6-004f-40dc-8251-6ac7a346b2eb None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:21.397834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791ad1c-ed82-4171-a4ad-1295d4b4de9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:21.398752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791ad1c-ed82-4171-a4ad-1295d4b4de9f None admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:21.399030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791ad1c-ed82-4171-a4ad-1295d4b4de9f None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:21.399299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791ad1c-ed82-4171-a4ad-1295d4b4de9f None admin] PATH_INFO: `/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/bba7d9205e694f2182d6091061d1219c/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:21.401217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791ad1c-ed82-4171-a4ad-1295d4b4de9f None admin] RBAC: Authorizing `identity:create_grant(project_id=dd62abd45e824417a0dda359d6fdabcd, user_id=bba7d9205e694f2182d6091061d1219c, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:21.402664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791ad1c-ed82-4171-a4ad-1295d4b4de9f None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:21.414986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 639/1273] 10.222.0.66 () {58 vars in 1439 bytes} [Fri Jul 1 19:43:21 2022] PUT /identity/v3/projects/dd62abd45e824417a0dda359d6fdabcd/users/bba7d9205e694f2182d6091061d1219c/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:43:23.415798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9f3ef-61fe-4009-95c2-e070c49709b7 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:23.416323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9f3ef-61fe-4009-95c2-e070c49709b7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:23.416323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9f3ef-61fe-4009-95c2-e070c49709b7 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:23.417723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 635/1274] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:43:23.422741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80780fe-db1f-434e-a578-a4d3e8efcaa0 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:23.423135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80780fe-db1f-434e-a578-a4d3e8efcaa0 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:23.423296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80780fe-db1f-434e-a578-a4d3e8efcaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:23.440152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-b80780fe-db1f-434e-a578-a4d3e8efcaa0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:43:23.441338 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b80780fe-db1f-434e-a578-a4d3e8efcaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:43:23.505932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 640/1275] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:43:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:43:23.530076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9f3ef-61fe-4009-95c2-e070c49709b7 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:23.629358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9f3ef-61fe-4009-95c2-e070c49709b7 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:23.645964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:23.664032 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e70057-3246-4799-aca9-ad17fe90eb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:23.664826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e70057-3246-4799-aca9-ad17fe90eb51 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:23.665104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e70057-3246-4799-aca9-ad17fe90eb51 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:23.665377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e70057-3246-4799-aca9-ad17fe90eb51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:23.675303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e70057-3246-4799-aca9-ad17fe90eb51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-85e70057-3246-4799-aca9-ad17fe90eb51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 19:43:23.683505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:23.684605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:23.685470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 636/1276] 10.222.0.66 () {56 vars in 1096 bytes} [Fri Jul 1 19:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:23.690818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80780fe-db1f-434e-a578-a4d3e8efcaa0 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:23.770086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:23.791200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80780fe-db1f-434e-a578-a4d3e8efcaa0 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:23.824345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d91d17-9be8-4bb8-b08d-0cfcb1ef6e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:23.824992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d91d17-9be8-4bb8-b08d-0cfcb1ef6e4c None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:23.825273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d91d17-9be8-4bb8-b08d-0cfcb1ef6e4c None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:23.825546 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d91d17-9be8-4bb8-b08d-0cfcb1ef6e4c None admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:23.826312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d91d17-9be8-4bb8-b08d-0cfcb1ef6e4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:23.826614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d91d17-9be8-4bb8-b08d-0cfcb1ef6e4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:23.827973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d91d17-9be8-4bb8-b08d-0cfcb1ef6e4c None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:23.836489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 641/1277] 10.222.0.66 () {56 vars in 1099 bytes} [Fri Jul 1 19:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:23.842187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e70057-3246-4799-aca9-ad17fe90eb51 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:23.938369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e70057-3246-4799-aca9-ad17fe90eb51 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:23.970710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccbdaa-b177-4d4a-817f-591eb58c96ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:23.971490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccbdaa-b177-4d4a-817f-591eb58c96ad None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:23.971767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccbdaa-b177-4d4a-817f-591eb58c96ad None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:23.972038 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccbdaa-b177-4d4a-817f-591eb58c96ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:23.982137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccbdaa-b177-4d4a-817f-591eb58c96ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:23.983565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccbdaa-b177-4d4a-817f-591eb58c96ad None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-68ccbdaa-b177-4d4a-817f-591eb58c96ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:24.000412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:24.001423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:24.002546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:24.002546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:24.002546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:24.002546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:24.002546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 637/1278] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:24.007976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d91d17-9be8-4bb8-b08d-0cfcb1ef6e4c None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:24.104795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d91d17-9be8-4bb8-b08d-0cfcb1ef6e4c None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:24.137905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfcc95-0144-459d-ba5f-aa6e91941886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:24.138546 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfcc95-0144-459d-ba5f-aa6e91941886 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:24.138822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfcc95-0144-459d-ba5f-aa6e91941886 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:24.139093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfcc95-0144-459d-ba5f-aa6e91941886 None admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:24.139796 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfcc95-0144-459d-ba5f-aa6e91941886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:24.140093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfcc95-0144-459d-ba5f-aa6e91941886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:24.141441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfcc95-0144-459d-ba5f-aa6e91941886 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:24.153400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 642/1279] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:43:24.159171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccbdaa-b177-4d4a-817f-591eb58c96ad None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:24.193568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:24.267098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccbdaa-b177-4d4a-817f-591eb58c96ad None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:24.300372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc372fd1-adc1-4dcd-ba7d-91007312b3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:24.301158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372fd1-adc1-4dcd-ba7d-91007312b3b8 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:24.301433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372fd1-adc1-4dcd-ba7d-91007312b3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:24.301737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc372fd1-adc1-4dcd-ba7d-91007312b3b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:24.310938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc372fd1-adc1-4dcd-ba7d-91007312b3b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:24.312380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc372fd1-adc1-4dcd-ba7d-91007312b3b8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-bc372fd1-adc1-4dcd-ba7d-91007312b3b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 19:43:24.327267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:24.328368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:24.334253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 638/1280] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:24.334999 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfcc95-0144-459d-ba5f-aa6e91941886 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:24.337980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:24.429570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfcc95-0144-459d-ba5f-aa6e91941886 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:24.462636 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b43b2-7234-4540-b55e-76717c43eeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:24.463225 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b43b2-7234-4540-b55e-76717c43eeff None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:24.463500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b43b2-7234-4540-b55e-76717c43eeff None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:24.463770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b43b2-7234-4540-b55e-76717c43eeff None admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:24.464586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b43b2-7234-4540-b55e-76717c43eeff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:24.464885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b43b2-7234-4540-b55e-76717c43eeff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:24.466308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b43b2-7234-4540-b55e-76717c43eeff None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:24.482431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 643/1281] 10.222.0.66 () {56 vars in 1143 bytes} [Fri Jul 1 19:43:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:24.488011 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc372fd1-adc1-4dcd-ba7d-91007312b3b8 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:24.577124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:24.586755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc372fd1-adc1-4dcd-ba7d-91007312b3b8 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:24.620509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791dd81-3cfa-4612-8c53-0dfa8b043d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:24.621160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791dd81-3cfa-4612-8c53-0dfa8b043d42 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:24.621440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791dd81-3cfa-4612-8c53-0dfa8b043d42 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:24.621744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791dd81-3cfa-4612-8c53-0dfa8b043d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:24.688108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791dd81-3cfa-4612-8c53-0dfa8b043d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:24.689548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791dd81-3cfa-4612-8c53-0dfa8b043d42 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:24.772809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:24.783368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 639/1282] 10.222.0.66 () {58 vars in 1282 bytes} [Fri Jul 1 19:43:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 19:43:24.789805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b43b2-7234-4540-b55e-76717c43eeff None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:24.849309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:24.897470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b43b2-7234-4540-b55e-76717c43eeff None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:24.937003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beae4c-cc7c-4a1e-a748-6bb106e162bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:24.937804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beae4c-cc7c-4a1e-a748-6bb106e162bf None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:24.938080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beae4c-cc7c-4a1e-a748-6bb106e162bf None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:24.938349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beae4c-cc7c-4a1e-a748-6bb106e162bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:24.948429 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44beae4c-cc7c-4a1e-a748-6bb106e162bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:24.949881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44beae4c-cc7c-4a1e-a748-6bb106e162bf None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-44beae4c-cc7c-4a1e-a748-6bb106e162bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 19:43:24.967332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:24.968382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:24.969438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 19:43:24.969438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 19:43:24.969438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 19:43:24.969438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:24.969438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 644/1283] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 19:43:24.974228 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791dd81-3cfa-4612-8c53-0dfa8b043d42 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:25.071769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791dd81-3cfa-4612-8c53-0dfa8b043d42 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:25.104935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35fa6d8-cdf1-42fa-b89c-a211080a2d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:25.105601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fa6d8-cdf1-42fa-b89c-a211080a2d85 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:25.105884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fa6d8-cdf1-42fa-b89c-a211080a2d85 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:25.106156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fa6d8-cdf1-42fa-b89c-a211080a2d85 None admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:25.106863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35fa6d8-cdf1-42fa-b89c-a211080a2d85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:25.107160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35fa6d8-cdf1-42fa-b89c-a211080a2d85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:25.108515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35fa6d8-cdf1-42fa-b89c-a211080a2d85 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:25.119281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 640/1284] 10.222.0.66 () {56 vars in 1109 bytes} [Fri Jul 1 19:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:25.124849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beae4c-cc7c-4a1e-a748-6bb106e162bf None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:25.229434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beae4c-cc7c-4a1e-a748-6bb106e162bf None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:25.243344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:25.262949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593633af-5275-4ec7-bdb3-eb3f77b10c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:25.263721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593633af-5275-4ec7-bdb3-eb3f77b10c53 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:25.264004 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593633af-5275-4ec7-bdb3-eb3f77b10c53 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:25.264272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593633af-5275-4ec7-bdb3-eb3f77b10c53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:25.270780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593633af-5275-4ec7-bdb3-eb3f77b10c53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:25.272224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593633af-5275-4ec7-bdb3-eb3f77b10c53 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application [None req-593633af-5275-4ec7-bdb3-eb3f77b10c53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 19:43:25.286478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:25.287509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:25.289132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 19:43:25.289132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 19:43:25.289132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 19:43:25.289132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 19:43:25.289132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 19:43:25.289132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 19:43:25.289132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 19:43:25.289132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: ERROR keystone.server.flask.application Jul 01 19:43:25.289132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 645/1285] 10.222.0.66 () {56 vars in 1111 bytes} [Fri Jul 1 19:43:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:25.294109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35fa6d8-cdf1-42fa-b89c-a211080a2d85 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:25.342954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:25.391285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35fa6d8-cdf1-42fa-b89c-a211080a2d85 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:25.425438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8a063-94dd-4dbe-b33e-df53d60ee4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:25.426111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8a063-94dd-4dbe-b33e-df53d60ee4a3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:25.426394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8a063-94dd-4dbe-b33e-df53d60ee4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:25.426667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8a063-94dd-4dbe-b33e-df53d60ee4a3 None admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:25.427446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8a063-94dd-4dbe-b33e-df53d60ee4a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:25.427748 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8a063-94dd-4dbe-b33e-df53d60ee4a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:25.429213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8a063-94dd-4dbe-b33e-df53d60ee4a3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:25.440648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 641/1286] 10.222.0.66 () {56 vars in 1148 bytes} [Fri Jul 1 19:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:25.447192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593633af-5275-4ec7-bdb3-eb3f77b10c53 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:25.547401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593633af-5275-4ec7-bdb3-eb3f77b10c53 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:25.581038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47249ec-0469-4e38-8c02-2c36488d7396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:25.581707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47249ec-0469-4e38-8c02-2c36488d7396 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:25.581985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47249ec-0469-4e38-8c02-2c36488d7396 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:25.582251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47249ec-0469-4e38-8c02-2c36488d7396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:25.583057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47249ec-0469-4e38-8c02-2c36488d7396 None admin] RBAC: Adding query filter params (role.id=f7da1306127e4efa845a2644d7be96d5, scope.project.id=dd62abd45e824417a0dda359d6fdabcd, user.id=bba7d9205e694f2182d6091061d1219c) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:25.583358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47249ec-0469-4e38-8c02-2c36488d7396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:25.584949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47249ec-0469-4e38-8c02-2c36488d7396 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:25.592098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 646/1287] 10.222.0.66 () {56 vars in 1374 bytes} [Fri Jul 1 19:43:25 2022] GET /identity/v3/role_assignments?user.id=bba7d9205e694f2182d6091061d1219c&scope.project.id=dd62abd45e824417a0dda359d6fdabcd&role.id=f7da1306127e4efa845a2644d7be96d5 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:27.644319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec73c13-5448-4420-b328-fed12228d71b None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:27.644822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec73c13-5448-4420-b328-fed12228d71b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:27.644943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec73c13-5448-4420-b328-fed12228d71b None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:27.646272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 642/1288] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:43:27.651993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a475880-7393-47ae-aec5-1c2a010b8b1b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:27.652406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a475880-7393-47ae-aec5-1c2a010b8b1b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:27.652543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a475880-7393-47ae-aec5-1c2a010b8b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:27.669295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7a475880-7393-47ae-aec5-1c2a010b8b1b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:43:27.670503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7a475880-7393-47ae-aec5-1c2a010b8b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:43:27.699610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:27.736547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 647/1289] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:43:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 19:43:27.759293 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec73c13-5448-4420-b328-fed12228d71b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:27.857523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec73c13-5448-4420-b328-fed12228d71b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:27.868604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:27.892929 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad703f19-4521-4c66-bcdc-b0c25878ed37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:27.893753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad703f19-4521-4c66-bcdc-b0c25878ed37 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:27.894036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad703f19-4521-4c66-bcdc-b0c25878ed37 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:27.894311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad703f19-4521-4c66-bcdc-b0c25878ed37 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:27.894958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad703f19-4521-4c66-bcdc-b0c25878ed37 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:27.896320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad703f19-4521-4c66-bcdc-b0c25878ed37 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-ad703f19-4521-4c66-bcdc-b0c25878ed37 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:27.909076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:27.910420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:27.911271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 643/1290] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 1 19:43:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:27.917002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a475880-7393-47ae-aec5-1c2a010b8b1b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:28.015595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a475880-7393-47ae-aec5-1c2a010b8b1b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:28.048966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616ccff-de22-46d8-8562-322597837b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:28.049643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616ccff-de22-46d8-8562-322597837b74 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:28.049925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616ccff-de22-46d8-8562-322597837b74 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:28.050195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616ccff-de22-46d8-8562-322597837b74 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:28.050883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616ccff-de22-46d8-8562-322597837b74 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:28.051185 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616ccff-de22-46d8-8562-322597837b74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:28.052538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616ccff-de22-46d8-8562-322597837b74 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:28.057296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 648/1291] 10.222.0.66 () {56 vars in 1116 bytes} [Fri Jul 1 19:43:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:28.063189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad703f19-4521-4c66-bcdc-b0c25878ed37 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:28.159198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad703f19-4521-4c66-bcdc-b0c25878ed37 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:28.196242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b210c-0998-41f0-ba2d-ce3e1ecac6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:28.196996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b210c-0998-41f0-ba2d-ce3e1ecac6d2 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:28.197276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b210c-0998-41f0-ba2d-ce3e1ecac6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:28.197557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b210c-0998-41f0-ba2d-ce3e1ecac6d2 None admin] PATH_INFO: `/v3/services` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:28.198434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b210c-0998-41f0-ba2d-ce3e1ecac6d2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:28.198772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b210c-0998-41f0-ba2d-ce3e1ecac6d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:28.200257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b210c-0998-41f0-ba2d-ce3e1ecac6d2 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:28.209632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 644/1292] 10.222.0.66 () {56 vars in 1116 bytes} [Fri Jul 1 19:43:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:30.260173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb61bc2-8f48-46f6-8517-be57bae9085b None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:30.260699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb61bc2-8f48-46f6-8517-be57bae9085b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:30.260699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb61bc2-8f48-46f6-8517-be57bae9085b None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:30.262095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 649/1293] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:43:30.267319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b952f-1a3e-4a1a-8aac-f18f7c2fe544 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:30.267743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b952f-1a3e-4a1a-8aac-f18f7c2fe544 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:30.267862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b952f-1a3e-4a1a-8aac-f18f7c2fe544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:30.285791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-078b952f-1a3e-4a1a-8aac-f18f7c2fe544 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:43:30.286953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-078b952f-1a3e-4a1a-8aac-f18f7c2fe544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:43:30.317286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:30.355100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 645/1294] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:43:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:43:30.378118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb61bc2-8f48-46f6-8517-be57bae9085b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:30.408971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:30.476306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb61bc2-8f48-46f6-8517-be57bae9085b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:30.509636 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b580640-f602-4216-b562-fefb86397dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:30.510341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b580640-f602-4216-b562-fefb86397dca None admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:30.510565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b580640-f602-4216-b562-fefb86397dca None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:30.510836 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b580640-f602-4216-b562-fefb86397dca None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:30.511566 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b580640-f602-4216-b562-fefb86397dca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:30.512915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b580640-f602-4216-b562-fefb86397dca None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:30.526365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 650/1295] 10.222.0.66 () {60 vars in 1141 bytes} [Fri Jul 1 19:43:30 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:43:32.566096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df16c7-f973-4d39-91a0-f74eed0f6883 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:32.566533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df16c7-f973-4d39-91a0-f74eed0f6883 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:32.566658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df16c7-f973-4d39-91a0-f74eed0f6883 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:32.568034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 646/1296] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:43:32.574022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc099b-e7f1-4dab-a8eb-3500316191ec None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:32.574414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc099b-e7f1-4dab-a8eb-3500316191ec None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:32.574574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc099b-e7f1-4dab-a8eb-3500316191ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:32.591278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9ebc099b-e7f1-4dab-a8eb-3500316191ec None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:43:32.592455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9ebc099b-e7f1-4dab-a8eb-3500316191ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:43:32.664500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 651/1297] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:43:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:43:32.688020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df16c7-f973-4d39-91a0-f74eed0f6883 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:32.785999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df16c7-f973-4d39-91a0-f74eed0f6883 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:32.818574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51e537-d0c7-40f6-a970-b4cb31fbbdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:32.819352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51e537-d0c7-40f6-a970-b4cb31fbbdd3 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:32.819633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51e537-d0c7-40f6-a970-b4cb31fbbdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:32.819906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51e537-d0c7-40f6-a970-b4cb31fbbdd3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:32.820645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51e537-d0c7-40f6-a970-b4cb31fbbdd3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:32.822536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51e537-d0c7-40f6-a970-b4cb31fbbdd3 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-2a51e537-d0c7-40f6-a970-b4cb31fbbdd3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:32.834157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:32.835349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:32.837109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 647/1298] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 1 19:43:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:32.841988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc099b-e7f1-4dab-a8eb-3500316191ec None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:32.860383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:32.938339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc099b-e7f1-4dab-a8eb-3500316191ec None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:32.974786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f349f-ab65-4e5c-8ea9-cd430d97a169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:32.975446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f349f-ab65-4e5c-8ea9-cd430d97a169 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:32.975888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f349f-ab65-4e5c-8ea9-cd430d97a169 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:32.976404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f349f-ab65-4e5c-8ea9-cd430d97a169 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:32.977208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f349f-ab65-4e5c-8ea9-cd430d97a169 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:32.977515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f349f-ab65-4e5c-8ea9-cd430d97a169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:32.978918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f349f-ab65-4e5c-8ea9-cd430d97a169 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:32.984140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 652/1299] 10.222.0.66 () {56 vars in 1116 bytes} [Fri Jul 1 19:43:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:32.990172 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51e537-d0c7-40f6-a970-b4cb31fbbdd3 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:33.021650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:33.090543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51e537-d0c7-40f6-a970-b4cb31fbbdd3 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:33.122502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfb133-f735-466d-874a-02a621ba9f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:33.123135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfb133-f735-466d-874a-02a621ba9f72 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:33.123409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfb133-f735-466d-874a-02a621ba9f72 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:33.123676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfb133-f735-466d-874a-02a621ba9f72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:33.124438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfb133-f735-466d-874a-02a621ba9f72 None admin] RBAC: Adding query filter params (interface=public, service_id=bd9a73751353459b891c04d9962dad41, region_id=RegionOne) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:33.124739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfb133-f735-466d-874a-02a621ba9f72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:33.126121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfb133-f735-466d-874a-02a621ba9f72 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:33.130927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 648/1300] 10.222.0.66 () {56 vars in 1251 bytes} [Fri Jul 1 19:43:32 2022] GET /identity/v3/endpoints?service_id=bd9a73751353459b891c04d9962dad41&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:33.137325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f349f-ab65-4e5c-8ea9-cd430d97a169 None admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:33.227843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:33.250418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f349f-ab65-4e5c-8ea9-cd430d97a169 None admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:33.284662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a26cec-b959-4311-8f27-9aa8a57ececa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:33.285316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a26cec-b959-4311-8f27-9aa8a57ececa None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:33.285621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a26cec-b959-4311-8f27-9aa8a57ececa None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:33.285906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a26cec-b959-4311-8f27-9aa8a57ececa None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:33.286581 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a26cec-b959-4311-8f27-9aa8a57ececa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:33.287970 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a26cec-b959-4311-8f27-9aa8a57ececa None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:33.294944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 653/1301] 10.222.0.66 () {56 vars in 1087 bytes} [Fri Jul 1 19:43:33 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:43:35.305172 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e8b6c-c86c-4922-85fa-1e8bf81cd57c None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:35.305778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e8b6c-c86c-4922-85fa-1e8bf81cd57c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:35.305778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e8b6c-c86c-4922-85fa-1e8bf81cd57c None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:35.307119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 649/1302] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:43:35.312292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5d9b3-1211-4df9-be6c-0fa5edb30e17 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:35.312806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5d9b3-1211-4df9-be6c-0fa5edb30e17 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:35.312907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5d9b3-1211-4df9-be6c-0fa5edb30e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:35.329476 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-67e5d9b3-1211-4df9-be6c-0fa5edb30e17 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:43:35.330686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-67e5d9b3-1211-4df9-be6c-0fa5edb30e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:43:35.395430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 654/1303] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:43:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 19:43:35.419787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e8b6c-c86c-4922-85fa-1e8bf81cd57c None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:35.514119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:35.521010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e8b6c-c86c-4922-85fa-1e8bf81cd57c None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:35.555357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c1c6b-00c9-4239-b68c-21085203a611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:35.556097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c1c6b-00c9-4239-b68c-21085203a611 None admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:35.556374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c1c6b-00c9-4239-b68c-21085203a611 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:35.556642 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c1c6b-00c9-4239-b68c-21085203a611 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:35.557290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c1c6b-00c9-4239-b68c-21085203a611 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:35.558700 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c1c6b-00c9-4239-b68c-21085203a611 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application [None req-920c1c6b-00c9-4239-b68c-21085203a611 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 19:43:35.571156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 19:43:35.572391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application with Lock( Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self._enter() Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application return self.creator() Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application created_value = creator( Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: ERROR keystone.server.flask.application Jul 01 19:43:35.573535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 650/1304] 10.222.0.66 () {56 vars in 1117 bytes} [Fri Jul 1 19:43:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 19:43:35.578503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5d9b3-1211-4df9-be6c-0fa5edb30e17 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:35.657832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:35.689563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5d9b3-1211-4df9-be6c-0fa5edb30e17 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:35.722733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18b34d-e65f-4594-81e7-ba14c6a8ed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:35.723382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18b34d-e65f-4594-81e7-ba14c6a8ed38 None admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:35.723661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18b34d-e65f-4594-81e7-ba14c6a8ed38 None admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:35.723933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18b34d-e65f-4594-81e7-ba14c6a8ed38 None admin] PATH_INFO: `/v3/services` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:35.724626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18b34d-e65f-4594-81e7-ba14c6a8ed38 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:43:35.724927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18b34d-e65f-4594-81e7-ba14c6a8ed38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:35.726984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18b34d-e65f-4594-81e7-ba14c6a8ed38 None admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:35.732194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 655/1305] 10.222.0.66 () {56 vars in 1116 bytes} [Fri Jul 1 19:43:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:43:35.737962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c1c6b-00c9-4239-b68c-21085203a611 None admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:43:35.834754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c1c6b-00c9-4239-b68c-21085203a611 None admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:43:35.867959 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9764651-b7b2-467a-90ca-a0cac6fe76b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:43:35.868589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9764651-b7b2-467a-90ca-a0cac6fe76b8 None admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:43:35.868871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9764651-b7b2-467a-90ca-a0cac6fe76b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:43:35.869147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9764651-b7b2-467a-90ca-a0cac6fe76b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:43:35.869909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9764651-b7b2-467a-90ca-a0cac6fe76b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:43:35.871268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9764651-b7b2-467a-90ca-a0cac6fe76b8 None admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:43:35.889597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 651/1306] 10.222.0.66 () {60 vars in 1144 bytes} [Fri Jul 1 19:43:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:43:45.729166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=30 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:43:45.885763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:39.279358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6423624-1912-41f5-8917-73dee55b10fc None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:39.279946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6423624-1912-41f5-8917-73dee55b10fc None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:39.279946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6423624-1912-41f5-8917-73dee55b10fc None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:39.287545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685d974-7358-422f-ad79-a4b25bb6800d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:39.287993 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685d974-7358-422f-ad79-a4b25bb6800d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:39.288183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685d974-7358-422f-ad79-a4b25bb6800d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:39.289865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 656/1307] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:44:39 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:44:39.388161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-1685d974-7358-422f-ad79-a4b25bb6800d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:44:39.389693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-1685d974-7358-422f-ad79-a4b25bb6800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:39.517215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 652/1308] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:44:39.576444 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53138c-b0d4-4412-b5bb-aaea107a4ebc None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:39.576736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53138c-b0d4-4412-b5bb-aaea107a4ebc None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:39.577017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53138c-b0d4-4412-b5bb-aaea107a4ebc None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:39.578678 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 657/1309] 10.222.0.66 () {54 vars in 964 bytes} [Fri Jul 1 19:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:44:39.587352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a7deb-658f-4b14-98cd-536c2fa46c32 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:39.587652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a7deb-658f-4b14-98cd-536c2fa46c32 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:39.587935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a7deb-658f-4b14-98cd-536c2fa46c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:39.705423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-8a1a7deb-658f-4b14-98cd-536c2fa46c32 None None] MFA Rules not processed for user `370f1c0f7af04a56bdf8fa452550380a`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:44:39.707584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8a1a7deb-658f-4b14-98cd-536c2fa46c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:39.734452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:39.796807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 653/1310] 10.222.0.66 () {58 vars in 1061 bytes} [Fri Jul 1 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:44:39.805984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53138c-b0d4-4412-b5bb-aaea107a4ebc None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:44:39.807888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7d53138c-b0d4-4412-b5bb-aaea107a4ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:39.929898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53138c-b0d4-4412-b5bb-aaea107a4ebc None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:44:40.009464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181eeaee-7a8f-4b83-9a71-c261e1d8c697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:44:40.010145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181eeaee-7a8f-4b83-9a71-c261e1d8c697 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:40.010427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181eeaee-7a8f-4b83-9a71-c261e1d8c697 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:40.010703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181eeaee-7a8f-4b83-9a71-c261e1d8c697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:40.012376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-181eeaee-7a8f-4b83-9a71-c261e1d8c697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:40.071813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:40.129425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181eeaee-7a8f-4b83-9a71-c261e1d8c697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:44:40.131491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181eeaee-7a8f-4b83-9a71-c261e1d8c697 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:44:40.151217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 658/1311] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:44:42.472749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b5fa-e3a0-4c6f-a6f3-0e0c295fb078 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:42.473314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b5fa-e3a0-4c6f-a6f3-0e0c295fb078 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:42.473314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b5fa-e3a0-4c6f-a6f3-0e0c295fb078 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:42.474872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 654/1312] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:44:42.480561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17adba3-16f9-4644-aa81-38067d09bc6f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:42.481002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17adba3-16f9-4644-aa81-38067d09bc6f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:42.481097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17adba3-16f9-4644-aa81-38067d09bc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:42.567895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a17adba3-16f9-4644-aa81-38067d09bc6f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:44:42.569276 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a17adba3-16f9-4644-aa81-38067d09bc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:42.677242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 659/1313] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:44:42.717015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939b5fa-e3a0-4c6f-a6f3-0e0c295fb078 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:44:42.718576 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a939b5fa-e3a0-4c6f-a6f3-0e0c295fb078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:42.823828 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939b5fa-e3a0-4c6f-a6f3-0e0c295fb078 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:44:42.898526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:42.901112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714e27f-df04-4b62-b773-ada04f640d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:44:42.901789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714e27f-df04-4b62-b773-ada04f640d69 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:42.902322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714e27f-df04-4b62-b773-ada04f640d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:42.902757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714e27f-df04-4b62-b773-ada04f640d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:42.904938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4714e27f-df04-4b62-b773-ada04f640d69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:43.015595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714e27f-df04-4b62-b773-ada04f640d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:44:43.017029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714e27f-df04-4b62-b773-ada04f640d69 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:44:43.036508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 655/1314] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:44:45.393305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc2af3-98f7-4394-8cf3-1f21918b4cae None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:45.393871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc2af3-98f7-4394-8cf3-1f21918b4cae None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:45.393964 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc2af3-98f7-4394-8cf3-1f21918b4cae None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:45.395822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 660/1315] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:44:45.401021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094b321-c2b5-4e8c-9ce4-b23914bbf28f None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:45.401413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094b321-c2b5-4e8c-9ce4-b23914bbf28f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:45.401591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094b321-c2b5-4e8c-9ce4-b23914bbf28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:45.466462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-3094b321-c2b5-4e8c-9ce4-b23914bbf28f None None] MFA Rules not processed for user `72cd343fe8fe49438f4bb8d66709e6ad`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:44:45.467665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3094b321-c2b5-4e8c-9ce4-b23914bbf28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:45.522938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:45.571792 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 656/1316] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:44:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:44:45.626530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8263467-4a40-4797-8b09-d504edea99d0 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:45.626822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8263467-4a40-4797-8b09-d504edea99d0 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:45.627099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8263467-4a40-4797-8b09-d504edea99d0 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:45.628484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 661/1317] 10.222.0.66 () {54 vars in 964 bytes} [Fri Jul 1 19:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:44:45.635310 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef639102-e837-4206-b1e3-ace4d01eea9b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:45.635594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef639102-e837-4206-b1e3-ace4d01eea9b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:45.635863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef639102-e837-4206-b1e3-ace4d01eea9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:45.713749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ef639102-e837-4206-b1e3-ace4d01eea9b None None] MFA Rules not processed for user `370f1c0f7af04a56bdf8fa452550380a`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:44:45.716251 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ef639102-e837-4206-b1e3-ace4d01eea9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:45.795586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 657/1318] 10.222.0.66 () {58 vars in 1061 bytes} [Fri Jul 1 19:44:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:44:45.805155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8263467-4a40-4797-8b09-d504edea99d0 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:44:45.806732 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e8263467-4a40-4797-8b09-d504edea99d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:45.816348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:45.911579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8263467-4a40-4797-8b09-d504edea99d0 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:44:45.984433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f6aec-0458-4be6-b661-5fc12fb64ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:44:45.984996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f6aec-0458-4be6-b661-5fc12fb64ded service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:45.985282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f6aec-0458-4be6-b661-5fc12fb64ded service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:45.985555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f6aec-0458-4be6-b661-5fc12fb64ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:45.987229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a60f6aec-0458-4be6-b661-5fc12fb64ded service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:46.047693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:46.097960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f6aec-0458-4be6-b661-5fc12fb64ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:44:46.099638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f6aec-0458-4be6-b661-5fc12fb64ded service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:44:46.118645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 662/1319] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:44:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:44:49.082147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f87bfb-fbf0-4905-b392-88e540c39828 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:49.083270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f87bfb-fbf0-4905-b392-88e540c39828 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:49.083270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f87bfb-fbf0-4905-b392-88e540c39828 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:49.084578 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 658/1320] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:44:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:44:49.089882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6497-f75d-4b81-8c91-9bc0aa7ba4e4 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:49.090254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6497-f75d-4b81-8c91-9bc0aa7ba4e4 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:49.090470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6497-f75d-4b81-8c91-9bc0aa7ba4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:49.154588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-248a6497-f75d-4b81-8c91-9bc0aa7ba4e4 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:44:49.155796 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-248a6497-f75d-4b81-8c91-9bc0aa7ba4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:49.255038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 663/1321] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:44:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:44:49.293343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f87bfb-fbf0-4905-b392-88e540c39828 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:44:49.294938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-75f87bfb-fbf0-4905-b392-88e540c39828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:49.344826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:49.397849 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f87bfb-fbf0-4905-b392-88e540c39828 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:44:49.469057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf326-c93c-46b3-aa02-57794f2e830b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:44:49.469725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf326-c93c-46b3-aa02-57794f2e830b service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:49.470008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf326-c93c-46b3-aa02-57794f2e830b service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:49.470281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf326-c93c-46b3-aa02-57794f2e830b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:49.471905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7dacf326-c93c-46b3-aa02-57794f2e830b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:49.567350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:49.580686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf326-c93c-46b3-aa02-57794f2e830b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:44:49.582166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf326-c93c-46b3-aa02-57794f2e830b service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:44:49.600790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 659/1322] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:44:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:44:52.359863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472170f-43b0-4792-822a-86379b46d7f9 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:52.360414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472170f-43b0-4792-822a-86379b46d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:52.360414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472170f-43b0-4792-822a-86379b46d7f9 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:52.361901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 664/1323] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:44:52.368205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca28fba-e9c8-491c-a8eb-34dd10088e3a None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:52.368665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca28fba-e9c8-491c-a8eb-34dd10088e3a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:52.368761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca28fba-e9c8-491c-a8eb-34dd10088e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:52.436919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9ca28fba-e9c8-491c-a8eb-34dd10088e3a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:44:52.438150 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9ca28fba-e9c8-491c-a8eb-34dd10088e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:52.548949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 660/1324] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:44:52.587278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472170f-43b0-4792-822a-86379b46d7f9 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:44:52.588893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b472170f-43b0-4792-822a-86379b46d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:52.608984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:52.697783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472170f-43b0-4792-822a-86379b46d7f9 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:44:52.772422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccee37-cf86-44d2-b90c-97afb6203343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:44:52.773066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccee37-cf86-44d2-b90c-97afb6203343 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:52.773344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccee37-cf86-44d2-b90c-97afb6203343 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:52.773641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccee37-cf86-44d2-b90c-97afb6203343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:52.775296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8fccee37-cf86-44d2-b90c-97afb6203343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:52.847872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:52.888966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccee37-cf86-44d2-b90c-97afb6203343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:44:52.890332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccee37-cf86-44d2-b90c-97afb6203343 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:44:52.909085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 665/1325] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:44:55.048429 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ab3b-cf52-4109-bc49-fe8309ada3a9 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:55.048950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ab3b-cf52-4109-bc49-fe8309ada3a9 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:55.048950 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ab3b-cf52-4109-bc49-fe8309ada3a9 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:55.050285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 661/1326] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:44:55.055665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6890bf-a439-4953-a7e1-914d6981ba12 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:55.056089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6890bf-a439-4953-a7e1-914d6981ba12 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:55.056202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6890bf-a439-4953-a7e1-914d6981ba12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:55.121941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-5c6890bf-a439-4953-a7e1-914d6981ba12 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:44:55.123182 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5c6890bf-a439-4953-a7e1-914d6981ba12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:55.223868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 666/1327] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:44:55.260278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6ab3b-cf52-4109-bc49-fe8309ada3a9 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:44:55.261881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0ef6ab3b-cf52-4109-bc49-fe8309ada3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:55.321318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:55.366961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6ab3b-cf52-4109-bc49-fe8309ada3a9 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:44:55.446145 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1175e-052e-4cec-af35-a8c39f0d0192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:44:55.446792 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1175e-052e-4cec-af35-a8c39f0d0192 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:55.447070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1175e-052e-4cec-af35-a8c39f0d0192 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:55.447339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1175e-052e-4cec-af35-a8c39f0d0192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:55.448975 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b4c1175e-052e-4cec-af35-a8c39f0d0192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:55.557816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:55.560457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1175e-052e-4cec-af35-a8c39f0d0192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:44:55.562385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1175e-052e-4cec-af35-a8c39f0d0192 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:44:55.589386 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 662/1328] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:44:57.695846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dfa18-ac96-4009-8f9d-c787b1c3fb77 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:57.696368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dfa18-ac96-4009-8f9d-c787b1c3fb77 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:57.696368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dfa18-ac96-4009-8f9d-c787b1c3fb77 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:57.697795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 667/1329] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:44:57.702900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cabc8e-054e-4f58-bd8c-9131d3f34222 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:57.703340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cabc8e-054e-4f58-bd8c-9131d3f34222 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:57.703459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cabc8e-054e-4f58-bd8c-9131d3f34222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:57.772758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-02cabc8e-054e-4f58-bd8c-9131d3f34222 None None] MFA Rules not processed for user `72cd343fe8fe49438f4bb8d66709e6ad`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:44:57.773912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-02cabc8e-054e-4f58-bd8c-9131d3f34222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:57.877102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 663/1330] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:44:57 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:44:57.915951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dfa18-ac96-4009-8f9d-c787b1c3fb77 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:44:57.917488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b82dfa18-ac96-4009-8f9d-c787b1c3fb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:57.950306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:58.024579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dfa18-ac96-4009-8f9d-c787b1c3fb77 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:44:58.100887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42a295-f06e-4874-b2a6-e8c0608178d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:44:58.101534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42a295-f06e-4874-b2a6-e8c0608178d5 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:44:58.101846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42a295-f06e-4874-b2a6-e8c0608178d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:44:58.102120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42a295-f06e-4874-b2a6-e8c0608178d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:44:58.103782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1f42a295-f06e-4874-b2a6-e8c0608178d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:44:58.181514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:44:58.213940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42a295-f06e-4874-b2a6-e8c0608178d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:44:58.215385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42a295-f06e-4874-b2a6-e8c0608178d5 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:44:58.246027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 668/1331] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:44:57 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:00.602086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd233a22-c4ac-4c21-a86d-c3eaaa03b8dd None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:00.602585 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd233a22-c4ac-4c21-a86d-c3eaaa03b8dd None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:00.602681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd233a22-c4ac-4c21-a86d-c3eaaa03b8dd None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:00.604003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 664/1332] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:00.609034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c35531-a634-49d4-bbf0-4062e7824bc3 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:00.609425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c35531-a634-49d4-bbf0-4062e7824bc3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:00.609596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c35531-a634-49d4-bbf0-4062e7824bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:00.673717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c7c35531-a634-49d4-bbf0-4062e7824bc3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:00.674934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c7c35531-a634-49d4-bbf0-4062e7824bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:00.771900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 669/1333] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:00.807691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd233a22-c4ac-4c21-a86d-c3eaaa03b8dd None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:00.809230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-cd233a22-c4ac-4c21-a86d-c3eaaa03b8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:00.877669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:00.912883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd233a22-c4ac-4c21-a86d-c3eaaa03b8dd None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:00.984714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d01a59-f864-4be1-b99e-81e96e8fb8ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:00.985362 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01a59-f864-4be1-b99e-81e96e8fb8ef service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:00.985680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01a59-f864-4be1-b99e-81e96e8fb8ef service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:00.985957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01a59-f864-4be1-b99e-81e96e8fb8ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:00.987599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b1d01a59-f864-4be1-b99e-81e96e8fb8ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:01.102510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d01a59-f864-4be1-b99e-81e96e8fb8ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:01.103964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d01a59-f864-4be1-b99e-81e96e8fb8ef service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:01.114778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:01.124203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 665/1334] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:03.184163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6574962-7626-4c1c-8dee-a6a328799d26 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:03.184717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6574962-7626-4c1c-8dee-a6a328799d26 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:03.184717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6574962-7626-4c1c-8dee-a6a328799d26 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:03.186139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 670/1335] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:03.191212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500566-9b46-4c4c-8f88-669cd8a5d05c None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:03.191590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500566-9b46-4c4c-8f88-669cd8a5d05c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:03.191849 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500566-9b46-4c4c-8f88-669cd8a5d05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:03.254346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d1500566-9b46-4c4c-8f88-669cd8a5d05c None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:03.255460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d1500566-9b46-4c4c-8f88-669cd8a5d05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:03.352079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 666/1336] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:03.392103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6574962-7626-4c1c-8dee-a6a328799d26 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:03.393676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d6574962-7626-4c1c-8dee-a6a328799d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:03.433349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:03.495562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6574962-7626-4c1c-8dee-a6a328799d26 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:03.568225 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93d730-ee66-450c-b018-b5fff3a4e3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:03.568875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d730-ee66-450c-b018-b5fff3a4e3fb service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:03.569160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d730-ee66-450c-b018-b5fff3a4e3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:03.569434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93d730-ee66-450c-b018-b5fff3a4e3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:03.571116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1c93d730-ee66-450c-b018-b5fff3a4e3fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:03.652302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:03.679479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93d730-ee66-450c-b018-b5fff3a4e3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:03.680918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93d730-ee66-450c-b018-b5fff3a4e3fb service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:03.700419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 671/1337] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:06.187223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d68e9-72a1-4713-aa8d-9d98525820d1 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:06.187887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d68e9-72a1-4713-aa8d-9d98525820d1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:06.187887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d68e9-72a1-4713-aa8d-9d98525820d1 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:06.189127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 667/1338] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:06.195225 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572c202-7df9-46b3-9cd4-85306f397038 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:06.195658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572c202-7df9-46b3-9cd4-85306f397038 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:06.195765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572c202-7df9-46b3-9cd4-85306f397038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:06.260591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d572c202-7df9-46b3-9cd4-85306f397038 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:06.261839 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d572c202-7df9-46b3-9cd4-85306f397038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:06.356113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 672/1339] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:06.393242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d68e9-72a1-4713-aa8d-9d98525820d1 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:06.394824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-809d68e9-72a1-4713-aa8d-9d98525820d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:06.470398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:06.498292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d68e9-72a1-4713-aa8d-9d98525820d1 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:06.571074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9e50e-bbfd-491b-a922-098c2b76ab32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:06.571723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9e50e-bbfd-491b-a922-098c2b76ab32 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:06.572004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9e50e-bbfd-491b-a922-098c2b76ab32 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:06.572279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9e50e-bbfd-491b-a922-098c2b76ab32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:06.573944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-92e9e50e-bbfd-491b-a922-098c2b76ab32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:06.681209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9e50e-bbfd-491b-a922-098c2b76ab32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:06.682565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9e50e-bbfd-491b-a922-098c2b76ab32 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:06.700580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 668/1340] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:10.794664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62dd3a-3efd-45e2-9a69-da0c4fdfeb1d None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:10.795191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62dd3a-3efd-45e2-9a69-da0c4fdfeb1d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:10.795191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62dd3a-3efd-45e2-9a69-da0c4fdfeb1d None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:10.796532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 673/1341] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:10.802338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cff4e-2d52-4a62-ba19-466187ce4e74 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:10.802778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cff4e-2d52-4a62-ba19-466187ce4e74 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:10.802954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cff4e-2d52-4a62-ba19-466187ce4e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:10.809033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:10.876053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-590cff4e-2d52-4a62-ba19-466187ce4e74 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:10.877262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-590cff4e-2d52-4a62-ba19-466187ce4e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:10.983176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 669/1342] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:11.023905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62dd3a-3efd-45e2-9a69-da0c4fdfeb1d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:11.026055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0f62dd3a-3efd-45e2-9a69-da0c4fdfeb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:11.075548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:11.125278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62dd3a-3efd-45e2-9a69-da0c4fdfeb1d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:11.129799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617041ae-ea61-4408-96f0-9307e9ccef9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:11.130625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617041ae-ea61-4408-96f0-9307e9ccef9c service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:11.130912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617041ae-ea61-4408-96f0-9307e9ccef9c service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:11.131209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617041ae-ea61-4408-96f0-9307e9ccef9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:11.140671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617041ae-ea61-4408-96f0-9307e9ccef9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:11.142127 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617041ae-ea61-4408-96f0-9307e9ccef9c service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:11.152027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 674/1343] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:13.805317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c703f-2650-41b0-8466-e7d7bf9770bd None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:13.805892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c703f-2650-41b0-8466-e7d7bf9770bd None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:13.805892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c703f-2650-41b0-8466-e7d7bf9770bd None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:13.807249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 670/1344] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:13.812684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bde0e-533a-46bb-b9a7-d81af89d6f83 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:13.813082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bde0e-533a-46bb-b9a7-d81af89d6f83 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:13.813370 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bde0e-533a-46bb-b9a7-d81af89d6f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:13.841281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-076bde0e-533a-46bb-b9a7-d81af89d6f83 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:13.842498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-076bde0e-533a-46bb-b9a7-d81af89d6f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:13.917882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 675/1345] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:13.965739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c703f-2650-41b0-8466-e7d7bf9770bd None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:13.977227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c703f-2650-41b0-8466-e7d7bf9770bd None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:13.980786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334b83d-70a9-490f-ba59-94e3fdb81186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:13.981441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b83d-70a9-490f-ba59-94e3fdb81186 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:13.981761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b83d-70a9-490f-ba59-94e3fdb81186 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:13.982038 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b83d-70a9-490f-ba59-94e3fdb81186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:13.992956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334b83d-70a9-490f-ba59-94e3fdb81186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:13.994452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334b83d-70a9-490f-ba59-94e3fdb81186 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:14.005293 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 671/1346] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:18.058406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aa340-db85-4cc3-ade2-234217a8c5fc None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:18.058961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aa340-db85-4cc3-ade2-234217a8c5fc None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:18.058961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aa340-db85-4cc3-ade2-234217a8c5fc None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:18.060399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 676/1347] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:18.066275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9940fd-1ce8-4b3c-88fe-caa6a2619500 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:18.066713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9940fd-1ce8-4b3c-88fe-caa6a2619500 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:18.066939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9940fd-1ce8-4b3c-88fe-caa6a2619500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:18.088930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-bb9940fd-1ce8-4b3c-88fe-caa6a2619500 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:18.090238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-bb9940fd-1ce8-4b3c-88fe-caa6a2619500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:18.098266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 672/1348] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:18.136374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aa340-db85-4cc3-ade2-234217a8c5fc None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:18.147769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aa340-db85-4cc3-ade2-234217a8c5fc None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:18.151500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb4885-1b29-48eb-9334-ec76cf8bf133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:18.152137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb4885-1b29-48eb-9334-ec76cf8bf133 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:18.152419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb4885-1b29-48eb-9334-ec76cf8bf133 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:18.152709 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb4885-1b29-48eb-9334-ec76cf8bf133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:18.162641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb4885-1b29-48eb-9334-ec76cf8bf133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:18.164121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb4885-1b29-48eb-9334-ec76cf8bf133 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:18.174063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 677/1349] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:45:23.340105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bd4a7-91ca-45bb-9a0f-4804b5e47da3 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:23.341730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bd4a7-91ca-45bb-9a0f-4804b5e47da3 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:23.341730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bd4a7-91ca-45bb-9a0f-4804b5e47da3 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:23.342412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 673/1350] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:23.347980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cc622-07e4-4afc-9d12-c0e57853a316 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:23.354427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cc622-07e4-4afc-9d12-c0e57853a316 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:23.354873 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cc622-07e4-4afc-9d12-c0e57853a316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:23.480027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a58cc622-07e4-4afc-9d12-c0e57853a316 None None] MFA Rules not processed for user `bba7d9205e694f2182d6091061d1219c`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:23.481389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a58cc622-07e4-4afc-9d12-c0e57853a316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:23.528363 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:23.619296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 678/1351] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:45:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:45:27.850193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c708248-9774-4550-9fc4-98caee642868 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:27.851050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c708248-9774-4550-9fc4-98caee642868 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:27.851326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c708248-9774-4550-9fc4-98caee642868 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:27.852885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 674/1352] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:27.858135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e761ee-c9a2-4aa8-ab78-c94a6f7cfd1c None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:27.858965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e761ee-c9a2-4aa8-ab78-c94a6f7cfd1c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:27.859473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e761ee-c9a2-4aa8-ab78-c94a6f7cfd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:27.881622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-07e761ee-c9a2-4aa8-ab78-c94a6f7cfd1c None None] MFA Rules not processed for user `bba7d9205e694f2182d6091061d1219c`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:27.883200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-07e761ee-c9a2-4aa8-ab78-c94a6f7cfd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:27.894412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 679/1353] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:45:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:28.083732 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=72 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:29.328652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a92145-850b-4bfc-b13a-5c562d07bf00 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:29.329287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a92145-850b-4bfc-b13a-5c562d07bf00 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:29.329287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a92145-850b-4bfc-b13a-5c562d07bf00 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:29.331392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 675/1354] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:29.336502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a18201-13e5-405d-9562-5462805b7f7d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:29.336961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a18201-13e5-405d-9562-5462805b7f7d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:29.337059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a18201-13e5-405d-9562-5462805b7f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:29.355799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-35a18201-13e5-405d-9562-5462805b7f7d None None] MFA Rules not processed for user `bba7d9205e694f2182d6091061d1219c`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:29.356989 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-35a18201-13e5-405d-9562-5462805b7f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:29.365395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 680/1355] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:45:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:32.946630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e902d5-f25e-4bd9-bf93-94a4ab1675af None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:32.947182 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e902d5-f25e-4bd9-bf93-94a4ab1675af None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:32.947738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e902d5-f25e-4bd9-bf93-94a4ab1675af None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:32.949230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 676/1356] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:32.956309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ee434-0d3a-444b-b2e4-4de6a4b36993 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:32.957246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ee434-0d3a-444b-b2e4-4de6a4b36993 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:32.958184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ee434-0d3a-444b-b2e4-4de6a4b36993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:32.980846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-3f7ee434-0d3a-444b-b2e4-4de6a4b36993 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:32.982557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3f7ee434-0d3a-444b-b2e4-4de6a4b36993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:32.998958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 681/1357] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:33.056947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aeb444-2586-4a18-b4c7-a930e45ea887 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:33.057252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aeb444-2586-4a18-b4c7-a930e45ea887 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:33.058698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aeb444-2586-4a18-b4c7-a930e45ea887 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:33.060078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 677/1358] 10.222.0.66 () {54 vars in 961 bytes} [Fri Jul 1 19:45:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:33.069239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4916b2d-fcea-4a52-bafa-1d9839b68890 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:33.069921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4916b2d-fcea-4a52-bafa-1d9839b68890 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:33.070568 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4916b2d-fcea-4a52-bafa-1d9839b68890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:33.124863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a4916b2d-fcea-4a52-bafa-1d9839b68890 None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:33.126025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a4916b2d-fcea-4a52-bafa-1d9839b68890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:33.218509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 682/1359] 10.222.0.66 () {58 vars in 1058 bytes} [Fri Jul 1 19:45:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:33.227262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aeb444-2586-4a18-b4c7-a930e45ea887 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:33.243756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aeb444-2586-4a18-b4c7-a930e45ea887 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:33.249726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37622aac-b7df-4dcf-88ad-0481fc63f52f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:33.250451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622aac-b7df-4dcf-88ad-0481fc63f52f service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:33.250840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622aac-b7df-4dcf-88ad-0481fc63f52f service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:33.251188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622aac-b7df-4dcf-88ad-0481fc63f52f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:33.261348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37622aac-b7df-4dcf-88ad-0481fc63f52f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:33.263049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37622aac-b7df-4dcf-88ad-0481fc63f52f service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:33.273851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 678/1360] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:35.559182 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb832d-fcb9-45d0-a4be-4ff430a94f87 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:35.559684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb832d-fcb9-45d0-a4be-4ff430a94f87 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:35.559789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb832d-fcb9-45d0-a4be-4ff430a94f87 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:35.561153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 683/1361] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:35.566858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9a46-6c19-4935-8eb5-a6ce496cc509 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:35.567303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9a46-6c19-4935-8eb5-a6ce496cc509 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:35.567417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9a46-6c19-4935-8eb5-a6ce496cc509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:35.586721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-652e9a46-6c19-4935-8eb5-a6ce496cc509 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:35.587942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-652e9a46-6c19-4935-8eb5-a6ce496cc509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:35.597650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 679/1362] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:35.641792 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb832d-fcb9-45d0-a4be-4ff430a94f87 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:35.657017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb832d-fcb9-45d0-a4be-4ff430a94f87 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:35.663301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2120b36-2d13-4c88-91d9-ec75f85ab58b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:35.664096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2120b36-2d13-4c88-91d9-ec75f85ab58b service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:35.664388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2120b36-2d13-4c88-91d9-ec75f85ab58b service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:35.664660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2120b36-2d13-4c88-91d9-ec75f85ab58b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:35.673376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2120b36-2d13-4c88-91d9-ec75f85ab58b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:35.675090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2120b36-2d13-4c88-91d9-ec75f85ab58b service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:35.686102 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 684/1363] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:38.072994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159c7d0-6d6f-4f7c-89b0-ccff7c8ad8a2 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:38.074468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159c7d0-6d6f-4f7c-89b0-ccff7c8ad8a2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:38.074468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159c7d0-6d6f-4f7c-89b0-ccff7c8ad8a2 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:38.074878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 680/1364] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:38.081773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba774d6-e4d0-40a2-ac41-5104592035b0 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:38.081773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba774d6-e4d0-40a2-ac41-5104592035b0 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:38.082273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba774d6-e4d0-40a2-ac41-5104592035b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:38.102874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7ba774d6-e4d0-40a2-ac41-5104592035b0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:38.104078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7ba774d6-e4d0-40a2-ac41-5104592035b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:38.113780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 685/1365] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:38.169609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba4e9-e1c7-48d7-9c36-b4d655b98d60 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:38.169609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba4e9-e1c7-48d7-9c36-b4d655b98d60 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:38.169609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ba4e9-e1c7-48d7-9c36-b4d655b98d60 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:38.170307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 681/1366] 10.222.0.66 () {54 vars in 961 bytes} [Fri Jul 1 19:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:38.177315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ee452-0638-420f-a724-c9a349e4632f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:38.177541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ee452-0638-420f-a724-c9a349e4632f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:38.177854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ee452-0638-420f-a724-c9a349e4632f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:38.201017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ac5ee452-0638-420f-a724-c9a349e4632f None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:38.202465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ac5ee452-0638-420f-a724-c9a349e4632f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:38.213127 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 686/1367] 10.222.0.66 () {58 vars in 1058 bytes} [Fri Jul 1 19:45:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:38.222066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba4e9-e1c7-48d7-9c36-b4d655b98d60 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:38.245147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ba4e9-e1c7-48d7-9c36-b4d655b98d60 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:38.250687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ed0f5-1eee-49d1-9429-a65b080c8e4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:38.251325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ed0f5-1eee-49d1-9429-a65b080c8e4b service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:38.251604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ed0f5-1eee-49d1-9429-a65b080c8e4b service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:38.251926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ed0f5-1eee-49d1-9429-a65b080c8e4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:38.260150 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ed0f5-1eee-49d1-9429-a65b080c8e4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:38.261761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ed0f5-1eee-49d1-9429-a65b080c8e4b service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:38.271496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 682/1368] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:40.394708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe755db-3319-4f4c-a1d4-1a3d2df1b950 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:40.395595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe755db-3319-4f4c-a1d4-1a3d2df1b950 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:40.396052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe755db-3319-4f4c-a1d4-1a3d2df1b950 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:40.397627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 687/1369] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:40.403004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf2906-3b80-4d27-8393-6f1001169af6 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:40.404028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf2906-3b80-4d27-8393-6f1001169af6 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:40.404492 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf2906-3b80-4d27-8393-6f1001169af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:40.422569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-29bf2906-3b80-4d27-8393-6f1001169af6 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:40.424159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-29bf2906-3b80-4d27-8393-6f1001169af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:40.432360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 683/1370] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:45:40.477199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe755db-3319-4f4c-a1d4-1a3d2df1b950 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:40.492355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe755db-3319-4f4c-a1d4-1a3d2df1b950 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:40.498027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86098c0-454a-4d09-b3d4-b9d84ac0162f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:40.498897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86098c0-454a-4d09-b3d4-b9d84ac0162f service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:40.499349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86098c0-454a-4d09-b3d4-b9d84ac0162f service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:40.499774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86098c0-454a-4d09-b3d4-b9d84ac0162f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:40.510587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86098c0-454a-4d09-b3d4-b9d84ac0162f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:40.513037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86098c0-454a-4d09-b3d4-b9d84ac0162f service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:40.529293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 688/1371] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:41.312228 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4adc5b-3e19-43ea-a7bb-8b4e30b47bc9 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:41.312522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4adc5b-3e19-43ea-a7bb-8b4e30b47bc9 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:41.312796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4adc5b-3e19-43ea-a7bb-8b4e30b47bc9 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:41.314215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 684/1372] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:41.321162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ed831-f5db-4f19-8036-57401e724ae8 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:41.321461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ed831-f5db-4f19-8036-57401e724ae8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:41.321776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ed831-f5db-4f19-8036-57401e724ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:41.340207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-af5ed831-f5db-4f19-8036-57401e724ae8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:41.341431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-af5ed831-f5db-4f19-8036-57401e724ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:41.362982 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 689/1373] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:41.692627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6012132-c94d-4963-9425-97ef675aa707 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:41.693237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6012132-c94d-4963-9425-97ef675aa707 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:41.693237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6012132-c94d-4963-9425-97ef675aa707 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:41.694709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 685/1374] 10.222.0.66 () {54 vars in 963 bytes} [Fri Jul 1 19:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:41.699659 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371ae2e-0871-478a-9dfc-e4f83a03c516 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:41.700158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371ae2e-0871-478a-9dfc-e4f83a03c516 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:41.700325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371ae2e-0871-478a-9dfc-e4f83a03c516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:41.709685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:41.756557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7371ae2e-0871-478a-9dfc-e4f83a03c516 None None] MFA Rules not processed for user `a4b0e6ea57af454cbc8f5bd180bc052e`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:41.757803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7371ae2e-0871-478a-9dfc-e4f83a03c516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:41.823674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 690/1375] 10.222.0.66 () {58 vars in 1060 bytes} [Fri Jul 1 19:45:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:41.831458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6012132-c94d-4963-9425-97ef675aa707 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:41.842679 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6012132-c94d-4963-9425-97ef675aa707 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:41.847530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197221a3-9c16-4c5c-be93-607514b0915b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:41.848622 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197221a3-9c16-4c5c-be93-607514b0915b service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:41.849091 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197221a3-9c16-4c5c-be93-607514b0915b service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:41.849551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197221a3-9c16-4c5c-be93-607514b0915b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:41.865474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197221a3-9c16-4c5c-be93-607514b0915b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:41.866941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197221a3-9c16-4c5c-be93-607514b0915b service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:41.877912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 686/1376] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:42.643631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719879bd-960f-4045-914d-4f00fb71d407 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:42.643934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719879bd-960f-4045-914d-4f00fb71d407 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:42.644376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719879bd-960f-4045-914d-4f00fb71d407 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:42.646193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 691/1377] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:42.651282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3ddf8-f3da-417e-a7c8-48b8e3b35cba None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:42.651577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3ddf8-f3da-417e-a7c8-48b8e3b35cba None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:42.651860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3ddf8-f3da-417e-a7c8-48b8e3b35cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:42.671383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d9c3ddf8-f3da-417e-a7c8-48b8e3b35cba None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:42.672585 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d9c3ddf8-f3da-417e-a7c8-48b8e3b35cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:42.684965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 687/1378] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:42.731389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719879bd-960f-4045-914d-4f00fb71d407 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:42.745892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719879bd-960f-4045-914d-4f00fb71d407 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:42.751528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9adead-baf5-4753-b2c7-f0632b03890e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:42.752698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9adead-baf5-4753-b2c7-f0632b03890e service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:42.753123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9adead-baf5-4753-b2c7-f0632b03890e service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:42.753403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9adead-baf5-4753-b2c7-f0632b03890e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:42.761783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9adead-baf5-4753-b2c7-f0632b03890e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:42.763387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9adead-baf5-4753-b2c7-f0632b03890e service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:42.773516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 692/1379] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:44.874566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f53af-b93f-4ea7-bd1e-772f105e57ff None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:44.875084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f53af-b93f-4ea7-bd1e-772f105e57ff None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:44.875182 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f53af-b93f-4ea7-bd1e-772f105e57ff None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:44.876479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 688/1380] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:44.881683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66cf2-b9c8-43c4-b844-a2a56aecd6e2 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:44.882076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66cf2-b9c8-43c4-b844-a2a56aecd6e2 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:44.882232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66cf2-b9c8-43c4-b844-a2a56aecd6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:44.918744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a1f66cf2-b9c8-43c4-b844-a2a56aecd6e2 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:44.920831 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a1f66cf2-b9c8-43c4-b844-a2a56aecd6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:44.936804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 693/1381] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:44.983484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f53af-b93f-4ea7-bd1e-772f105e57ff None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:44.998538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f53af-b93f-4ea7-bd1e-772f105e57ff None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:45.004211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977acddd-7e10-4d91-8a5b-eab1c3f52d18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:45.004998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977acddd-7e10-4d91-8a5b-eab1c3f52d18 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:45.005348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977acddd-7e10-4d91-8a5b-eab1c3f52d18 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:45.005790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977acddd-7e10-4d91-8a5b-eab1c3f52d18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:45.014449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977acddd-7e10-4d91-8a5b-eab1c3f52d18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:45.016020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977acddd-7e10-4d91-8a5b-eab1c3f52d18 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:45.025539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 689/1382] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:47.113025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56668e02-665f-4338-a7ec-1e14c8810606 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:47.113565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56668e02-665f-4338-a7ec-1e14c8810606 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:47.113565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56668e02-665f-4338-a7ec-1e14c8810606 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:47.114966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 694/1383] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:47.120529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da9505-c45d-4862-a71a-933e4742aac7 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:47.120884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da9505-c45d-4862-a71a-933e4742aac7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:47.120884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da9505-c45d-4862-a71a-933e4742aac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:47.139424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-15da9505-c45d-4862-a71a-933e4742aac7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:47.140674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-15da9505-c45d-4862-a71a-933e4742aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:47.149463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 690/1384] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:47.192438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56668e02-665f-4338-a7ec-1e14c8810606 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:47.209861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56668e02-665f-4338-a7ec-1e14c8810606 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:47.215331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04673022-27b5-4a46-8de8-30232842d44e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:47.215971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04673022-27b5-4a46-8de8-30232842d44e service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:47.216250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04673022-27b5-4a46-8de8-30232842d44e service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:47.216522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04673022-27b5-4a46-8de8-30232842d44e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:47.225213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04673022-27b5-4a46-8de8-30232842d44e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:47.226808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04673022-27b5-4a46-8de8-30232842d44e service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:47.236598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 695/1385] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:49.310195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc71a7-fa0c-4473-a8dd-efcac7b15811 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:49.310646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc71a7-fa0c-4473-a8dd-efcac7b15811 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:49.310812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc71a7-fa0c-4473-a8dd-efcac7b15811 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:49.318003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55af7-8321-4bb0-b90f-b8d614a810c4 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:49.318003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55af7-8321-4bb0-b90f-b8d614a810c4 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:49.318704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55af7-8321-4bb0-b90f-b8d614a810c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:49.324769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 691/1386] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:49 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:49.356100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-66f55af7-8321-4bb0-b90f-b8d614a810c4 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:49.357693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-66f55af7-8321-4bb0-b90f-b8d614a810c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:49.368514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 696/1387] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:49.417840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc71a7-fa0c-4473-a8dd-efcac7b15811 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:49.432832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc71a7-fa0c-4473-a8dd-efcac7b15811 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:49.439088 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb602a85-3235-4477-a004-81c0ea3e9c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:49.440258 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb602a85-3235-4477-a004-81c0ea3e9c1c service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:49.440859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb602a85-3235-4477-a004-81c0ea3e9c1c service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:49.440859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb602a85-3235-4477-a004-81c0ea3e9c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:49.451612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb602a85-3235-4477-a004-81c0ea3e9c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:49.453203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb602a85-3235-4477-a004-81c0ea3e9c1c service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:49.463914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 692/1388] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:51.615446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ee5e1-1a33-4f03-9cd4-64f4ecd97341 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:51.616000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ee5e1-1a33-4f03-9cd4-64f4ecd97341 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:51.616000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ee5e1-1a33-4f03-9cd4-64f4ecd97341 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:51.617307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 697/1389] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:45:51.622933 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478628fa-760c-49f6-a299-5db77c22de11 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:51.623374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478628fa-760c-49f6-a299-5db77c22de11 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:51.623634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478628fa-760c-49f6-a299-5db77c22de11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:51.644900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-478628fa-760c-49f6-a299-5db77c22de11 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:51.646219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-478628fa-760c-49f6-a299-5db77c22de11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:51.654968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 693/1390] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:45:51.701225 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ee5e1-1a33-4f03-9cd4-64f4ecd97341 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:51.717965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ee5e1-1a33-4f03-9cd4-64f4ecd97341 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:51.723874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597c831-e9f8-4a41-b97d-791cf0270d2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:51.724546 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597c831-e9f8-4a41-b97d-791cf0270d2d service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:51.724831 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597c831-e9f8-4a41-b97d-791cf0270d2d service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:51.725107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597c831-e9f8-4a41-b97d-791cf0270d2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:51.733606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597c831-e9f8-4a41-b97d-791cf0270d2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:51.735216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597c831-e9f8-4a41-b97d-791cf0270d2d service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:51.744871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 698/1391] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:53.882786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7312e-d785-4259-b726-95de8c5a8d6b None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:53.883330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7312e-d785-4259-b726-95de8c5a8d6b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:53.883579 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7312e-d785-4259-b726-95de8c5a8d6b None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:53.885330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 694/1392] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:53.890646 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb9d6c-3ab1-4a87-8e0e-0b41b3176847 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:53.891229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb9d6c-3ab1-4a87-8e0e-0b41b3176847 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:53.891229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb9d6c-3ab1-4a87-8e0e-0b41b3176847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:53.909564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-fbdb9d6c-3ab1-4a87-8e0e-0b41b3176847 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:53.910798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fbdb9d6c-3ab1-4a87-8e0e-0b41b3176847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:53.918628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 699/1393] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:53.964951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7312e-d785-4259-b726-95de8c5a8d6b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:53.983983 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7312e-d785-4259-b726-95de8c5a8d6b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:53.992595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f91cf-4135-4452-8f01-9ecd215b7493 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:53.993565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f91cf-4135-4452-8f01-9ecd215b7493 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:53.994034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f91cf-4135-4452-8f01-9ecd215b7493 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:53.994584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f91cf-4135-4452-8f01-9ecd215b7493 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:54.008186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f91cf-4135-4452-8f01-9ecd215b7493 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:54.010945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f91cf-4135-4452-8f01-9ecd215b7493 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:54.026600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 695/1394] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:56.161709 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035c97d-7339-4a8a-8020-21aff82f616a None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:56.162248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035c97d-7339-4a8a-8020-21aff82f616a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:56.162248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035c97d-7339-4a8a-8020-21aff82f616a None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:56.163623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 700/1395] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:56.168863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb76ce3-201a-4966-ae78-d92bb39300fd None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:56.169306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb76ce3-201a-4966-ae78-d92bb39300fd None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:56.169489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb76ce3-201a-4966-ae78-d92bb39300fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:56.188261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-1fb76ce3-201a-4966-ae78-d92bb39300fd None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:56.189566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-1fb76ce3-201a-4966-ae78-d92bb39300fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:56.197353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 696/1396] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:56.241693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035c97d-7339-4a8a-8020-21aff82f616a None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:56.256214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035c97d-7339-4a8a-8020-21aff82f616a None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:56.268003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee6d02-8d47-41b4-b31e-cd083531b1c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:56.268652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee6d02-8d47-41b4-b31e-cd083531b1c1 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:56.268937 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee6d02-8d47-41b4-b31e-cd083531b1c1 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:56.269214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee6d02-8d47-41b4-b31e-cd083531b1c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:56.278079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee6d02-8d47-41b4-b31e-cd083531b1c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:56.279914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee6d02-8d47-41b4-b31e-cd083531b1c1 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:56.291010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 701/1397] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:57.229561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa3cf7-2944-4a8f-bca8-6c0e44b37f86 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:57.230139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa3cf7-2944-4a8f-bca8-6c0e44b37f86 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:57.230139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa3cf7-2944-4a8f-bca8-6c0e44b37f86 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:57.231519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 697/1398] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:57.237660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62713b5e-9741-4fc5-aea8-598a023e8d26 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:57.238103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62713b5e-9741-4fc5-aea8-598a023e8d26 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:57.238310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62713b5e-9741-4fc5-aea8-598a023e8d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:57.255881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-62713b5e-9741-4fc5-aea8-598a023e8d26 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:57.257073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-62713b5e-9741-4fc5-aea8-598a023e8d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:57.264974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 702/1399] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:57.324698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1efd3-4b6f-4c6c-b9e2-f517504a2fa6 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:57.325219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1efd3-4b6f-4c6c-b9e2-f517504a2fa6 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:57.326125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1efd3-4b6f-4c6c-b9e2-f517504a2fa6 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:57.328312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 698/1400] 10.222.0.66 () {54 vars in 962 bytes} [Fri Jul 1 19:45:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:57.334908 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b5102-08cc-4971-93c3-4e78badf2728 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:57.335227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b5102-08cc-4971-93c3-4e78badf2728 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:57.335471 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b5102-08cc-4971-93c3-4e78badf2728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:57.395433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d78b5102-08cc-4971-93c3-4e78badf2728 None None] MFA Rules not processed for user `c022224b03a54ab489d95502c63a25dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:57.396621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d78b5102-08cc-4971-93c3-4e78badf2728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:57.430452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:57.463137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 703/1401] 10.222.0.66 () {58 vars in 1059 bytes} [Fri Jul 1 19:45:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:45:57.471384 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1efd3-4b6f-4c6c-b9e2-f517504a2fa6 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:57.482032 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1efd3-4b6f-4c6c-b9e2-f517504a2fa6 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:57.485645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b55772-60f5-4822-814b-e09d60eb891c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c022224b03a54ab489d95502c63a25dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:57.486788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b55772-60f5-4822-814b-e09d60eb891c service glance] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:57.487063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b55772-60f5-4822-814b-e09d60eb891c service glance] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:57.487333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b55772-60f5-4822-814b-e09d60eb891c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:57.496442 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b55772-60f5-4822-814b-e09d60eb891c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:57.497907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b55772-60f5-4822-814b-e09d60eb891c service glance] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:57.510325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 699/1402] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:45:58.221706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff775c-5528-484e-9d8a-a199182fcbb5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:58.222037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff775c-5528-484e-9d8a-a199182fcbb5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:58.222314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff775c-5528-484e-9d8a-a199182fcbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:58.270558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-96ff775c-5528-484e-9d8a-a199182fcbb5 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:58.271752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-96ff775c-5528-484e-9d8a-a199182fcbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:58.341351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 704/1403] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:45:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:58.368176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df96e-6ad2-498d-b3f0-9b04a71d52ad None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:58.368663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df96e-6ad2-498d-b3f0-9b04a71d52ad None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:58.368743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df96e-6ad2-498d-b3f0-9b04a71d52ad None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:58.370123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 700/1404] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:45:58.375338 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9cd1-b84d-4740-939f-553a9ee35ff2 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:58.375631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9cd1-b84d-4740-939f-553a9ee35ff2 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:58.375912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b9cd1-b84d-4740-939f-553a9ee35ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:58.383195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cacdb-10ba-4033-b242-5a2665f16c73 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:58.383486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cacdb-10ba-4033-b242-5a2665f16c73 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:58.383764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cacdb-10ba-4033-b242-5a2665f16c73 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:58.385152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 701/1405] 10.222.0.66 () {54 vars in 942 bytes} [Fri Jul 1 19:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:45:58.392000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf28b7e-e4f1-40ad-9467-3d5e9fa883fd None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:58.392291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf28b7e-e4f1-40ad-9467-3d5e9fa883fd None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:58.393262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf28b7e-e4f1-40ad-9467-3d5e9fa883fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:58.394954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-942b9cd1-b84d-4740-939f-553a9ee35ff2 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:58.396168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-942b9cd1-b84d-4740-939f-553a9ee35ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:58.398930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=61 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:45:58.405524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 705/1406] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:58.447051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-fcf28b7e-e4f1-40ad-9467-3d5e9fa883fd None None] MFA Rules not processed for user `c2bce9daec8c446d89eb6cc5950d6649`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:45:58.448388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-fcf28b7e-e4f1-40ad-9467-3d5e9fa883fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:45:58.450105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b9cd1-b84d-4740-939f-553a9ee35ff2 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:58.462745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b9cd1-b84d-4740-939f-553a9ee35ff2 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:58.469389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fec0c-8bd4-4022-bba6-55896175381d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:58.470134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fec0c-8bd4-4022-bba6-55896175381d service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:58.470454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fec0c-8bd4-4022-bba6-55896175381d service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:58.471239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fec0c-8bd4-4022-bba6-55896175381d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:58.480705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fec0c-8bd4-4022-bba6-55896175381d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:58.482526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fec0c-8bd4-4022-bba6-55896175381d service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:58.492713 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 706/1407] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:45:58.521468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 702/1408] 10.222.0.66 () {58 vars in 1039 bytes} [Fri Jul 1 19:45:58 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:45:58.531478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fec0c-8bd4-4022-bba6-55896175381d service nova] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:45:58.542502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fec0c-8bd4-4022-bba6-55896175381d service nova] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:45:58.548506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6fa89-2380-4ae3-8321-7b9cc53de369 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:45:58.549661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6fa89-2380-4ae3-8321-7b9cc53de369 service swift] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:45:58.550051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6fa89-2380-4ae3-8321-7b9cc53de369 service swift] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:45:58.550308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6fa89-2380-4ae3-8321-7b9cc53de369 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:45:58.559486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6fa89-2380-4ae3-8321-7b9cc53de369 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:45:58.567772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6fa89-2380-4ae3-8321-7b9cc53de369 service swift] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:45:58.573121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 707/1409] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:45:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:46:08.517813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=43 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:46:08.570030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=65 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:46:21.068715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bb93a-713d-405f-84d2-96d0cdabaf20 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:21.068715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bb93a-713d-405f-84d2-96d0cdabaf20 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:21.068715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13bb93a-713d-405f-84d2-96d0cdabaf20 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:21.068715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 703/1410] 10.222.0.66 () {54 vars in 918 bytes} [Fri Jul 1 19:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:21.073368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a93b9e-0d07-4b50-8e75-06945fec4a7a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:21.073830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a93b9e-0d07-4b50-8e75-06945fec4a7a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:21.073939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a93b9e-0d07-4b50-8e75-06945fec4a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:21.092736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-63a93b9e-0d07-4b50-8e75-06945fec4a7a None None] MFA Rules not processed for user `bba7d9205e694f2182d6091061d1219c`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:21.094010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-63a93b9e-0d07-4b50-8e75-06945fec4a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:21.103340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 708/1411] 10.222.0.66 () {58 vars in 1015 bytes} [Fri Jul 1 19:46:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:21.117841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d83828-fc6b-4d1e-a4fb-9faa588b8293 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:21.118124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d83828-fc6b-4d1e-a4fb-9faa588b8293 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:21.118397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d83828-fc6b-4d1e-a4fb-9faa588b8293 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:21.119801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 704/1412] 10.222.0.66 () {54 vars in 918 bytes} [Fri Jul 1 19:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:21.124506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753502b-78e3-424d-865e-924f7d11de0b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:21.124789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753502b-78e3-424d-865e-924f7d11de0b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:21.125061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753502b-78e3-424d-865e-924f7d11de0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:21.148755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9753502b-78e3-424d-865e-924f7d11de0b None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:21.150028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9753502b-78e3-424d-865e-924f7d11de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:21.159442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 709/1413] 10.222.0.66 () {58 vars in 1015 bytes} [Fri Jul 1 19:46:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:23.738408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543c726-af63-4813-a7fe-ec9543cbd153 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:23.738915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543c726-af63-4813-a7fe-ec9543cbd153 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:23.738915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543c726-af63-4813-a7fe-ec9543cbd153 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:23.740265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:23.745400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f57c5-5343-473d-9131-2f9b8c365ecb None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:23.745844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f57c5-5343-473d-9131-2f9b8c365ecb None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:23.745979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f57c5-5343-473d-9131-2f9b8c365ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:23.764144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a80f57c5-5343-473d-9131-2f9b8c365ecb None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:23.765333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a80f57c5-5343-473d-9131-2f9b8c365ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:23.773095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:23.817269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543c726-af63-4813-a7fe-ec9543cbd153 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:23.832961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543c726-af63-4813-a7fe-ec9543cbd153 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:23.838379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae78cc-0021-4422-bfe9-655d0eb8d517 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:23.839041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae78cc-0021-4422-bfe9-655d0eb8d517 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:23.839350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae78cc-0021-4422-bfe9-655d0eb8d517 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:23.839628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae78cc-0021-4422-bfe9-655d0eb8d517 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:23.848461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae78cc-0021-4422-bfe9-655d0eb8d517 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:23.850067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae78cc-0021-4422-bfe9-655d0eb8d517 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:23.859534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 706/1416] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:24.762097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859bd2-a266-40d4-a04d-786a965471d5 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:24.762561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859bd2-a266-40d4-a04d-786a965471d5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:24.762686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859bd2-a266-40d4-a04d-786a965471d5 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:24.764115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:24.770026 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a8b94-9c02-4f98-840d-66e9a9736331 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:24.770434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a8b94-9c02-4f98-840d-66e9a9736331 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:24.770563 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a8b94-9c02-4f98-840d-66e9a9736331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:24.788373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-362a8b94-9c02-4f98-840d-66e9a9736331 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:24.789542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-362a8b94-9c02-4f98-840d-66e9a9736331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:24.797151 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:24.948334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa859bd2-a266-40d4-a04d-786a965471d5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:24.963377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa859bd2-a266-40d4-a04d-786a965471d5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:24.968744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f022d8-541a-4fc4-a9c2-48f99a5eb035 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:24.969377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f022d8-541a-4fc4-a9c2-48f99a5eb035 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:24.969688 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f022d8-541a-4fc4-a9c2-48f99a5eb035 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:24.969963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f022d8-541a-4fc4-a9c2-48f99a5eb035 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:24.978188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f022d8-541a-4fc4-a9c2-48f99a5eb035 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:24.979753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f022d8-541a-4fc4-a9c2-48f99a5eb035 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:24.989804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 712/1419] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:25.872994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e345a-200f-4343-a0ca-42f17a30f7c0 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:25.873527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e345a-200f-4343-a0ca-42f17a30f7c0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:25.873527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e345a-200f-4343-a0ca-42f17a30f7c0 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:25.874970 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:25.880576 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf55821-4ad2-440b-971a-7c0c3fd42140 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:25.881000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf55821-4ad2-440b-971a-7c0c3fd42140 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:25.881133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf55821-4ad2-440b-971a-7c0c3fd42140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:25.899006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ebf55821-4ad2-440b-971a-7c0c3fd42140 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:25.900198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ebf55821-4ad2-440b-971a-7c0c3fd42140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:25.908618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:26.063177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e345a-200f-4343-a0ca-42f17a30f7c0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:26.077650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e345a-200f-4343-a0ca-42f17a30f7c0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:26.083863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e54f0d-4ac7-41de-8d29-1b427d3d255c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:26.084483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e54f0d-4ac7-41de-8d29-1b427d3d255c service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:26.084783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e54f0d-4ac7-41de-8d29-1b427d3d255c service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:26.085160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e54f0d-4ac7-41de-8d29-1b427d3d255c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:26.093008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e54f0d-4ac7-41de-8d29-1b427d3d255c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:26.094601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e54f0d-4ac7-41de-8d29-1b427d3d255c service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:26.104928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 709/1422] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:26.976074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e6bd8-4702-4af0-8259-79c49521e009 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:26.976653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e6bd8-4702-4af0-8259-79c49521e009 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:26.976653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e6bd8-4702-4af0-8259-79c49521e009 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:26.978072 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:26.983213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470bb27-27a2-4022-8ac5-2194e797264e None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:26.983612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470bb27-27a2-4022-8ac5-2194e797264e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:26.983777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470bb27-27a2-4022-8ac5-2194e797264e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:27.002234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d470bb27-27a2-4022-8ac5-2194e797264e None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:27.003437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d470bb27-27a2-4022-8ac5-2194e797264e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:27.011159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:27.188567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e6bd8-4702-4af0-8259-79c49521e009 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:27.202736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e6bd8-4702-4af0-8259-79c49521e009 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:27.208460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51199a27-9c10-4e19-9d33-d6c4ffddcfcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:27.209091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51199a27-9c10-4e19-9d33-d6c4ffddcfcf service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:27.209373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51199a27-9c10-4e19-9d33-d6c4ffddcfcf service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:27.209677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51199a27-9c10-4e19-9d33-d6c4ffddcfcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:27.217608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51199a27-9c10-4e19-9d33-d6c4ffddcfcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:27.219190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51199a27-9c10-4e19-9d33-d6c4ffddcfcf service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:27.229350 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 715/1425] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:28.109275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5ad6b-e2b6-43bf-8c1a-a0fa24ba319b None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:28.109837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5ad6b-e2b6-43bf-8c1a-a0fa24ba319b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:28.109837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5ad6b-e2b6-43bf-8c1a-a0fa24ba319b None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:28.111697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:28.116750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefadd8c-0cdf-4a62-a1e2-39d0e56e924d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:28.117137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefadd8c-0cdf-4a62-a1e2-39d0e56e924d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:28.117303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefadd8c-0cdf-4a62-a1e2-39d0e56e924d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:28.135008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-aefadd8c-0cdf-4a62-a1e2-39d0e56e924d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:28.136205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-aefadd8c-0cdf-4a62-a1e2-39d0e56e924d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:28.144659 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:28.295383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5ad6b-e2b6-43bf-8c1a-a0fa24ba319b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:28.309938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5ad6b-e2b6-43bf-8c1a-a0fa24ba319b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:28.315413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cdf28-e664-43da-a371-713617c4b5db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:28.316044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cdf28-e664-43da-a371-713617c4b5db service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:28.316325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cdf28-e664-43da-a371-713617c4b5db service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:28.316600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cdf28-e664-43da-a371-713617c4b5db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:28.324644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cdf28-e664-43da-a371-713617c4b5db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:28.326279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cdf28-e664-43da-a371-713617c4b5db service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:28.336149 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 712/1428] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:29.149326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db06195-a9cc-4117-9956-4e1743da0149 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:29.149941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db06195-a9cc-4117-9956-4e1743da0149 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:29.149941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db06195-a9cc-4117-9956-4e1743da0149 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:29.151265 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:29.156291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca8e38-0534-462c-be8b-ca35b143de0d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:29.156733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca8e38-0534-462c-be8b-ca35b143de0d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:29.156855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca8e38-0534-462c-be8b-ca35b143de0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:29.175644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-0dca8e38-0534-462c-be8b-ca35b143de0d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:29.176831 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0dca8e38-0534-462c-be8b-ca35b143de0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:29.184518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:29.344371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db06195-a9cc-4117-9956-4e1743da0149 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:29.359159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db06195-a9cc-4117-9956-4e1743da0149 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:29.364727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412a496-238e-487d-851e-02928614e908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:29.365356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412a496-238e-487d-851e-02928614e908 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:29.365670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412a496-238e-487d-851e-02928614e908 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:29.365953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412a496-238e-487d-851e-02928614e908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:29.373990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412a496-238e-487d-851e-02928614e908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:29.375570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412a496-238e-487d-851e-02928614e908 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:29.385565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 718/1431] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:30.191600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3359cc-709e-4af4-b364-8844ea3efcf5 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:30.192147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3359cc-709e-4af4-b364-8844ea3efcf5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:30.192266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3359cc-709e-4af4-b364-8844ea3efcf5 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:30.193553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:30.198535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e672981-5de3-45d9-a5cb-bf31b8ceb396 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:30.198956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e672981-5de3-45d9-a5cb-bf31b8ceb396 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:30.199096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e672981-5de3-45d9-a5cb-bf31b8ceb396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:30.216925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-0e672981-5de3-45d9-a5cb-bf31b8ceb396 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:30.218144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0e672981-5de3-45d9-a5cb-bf31b8ceb396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:30.226771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:30.378369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3359cc-709e-4af4-b364-8844ea3efcf5 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:30.393103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3359cc-709e-4af4-b364-8844ea3efcf5 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:30.398676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c4dd2-7444-48e1-9011-d41c3005e9d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:30.399316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c4dd2-7444-48e1-9011-d41c3005e9d9 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:30.399603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c4dd2-7444-48e1-9011-d41c3005e9d9 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:30.399882 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c4dd2-7444-48e1-9011-d41c3005e9d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:30.409478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c4dd2-7444-48e1-9011-d41c3005e9d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:30.411086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c4dd2-7444-48e1-9011-d41c3005e9d9 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:30.420863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 715/1434] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:31.226645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68f079-a114-46a6-a50a-a4c8b5db8c5e None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:31.227173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68f079-a114-46a6-a50a-a4c8b5db8c5e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:31.227173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68f079-a114-46a6-a50a-a4c8b5db8c5e None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:31.228579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:31.233740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9a6b-8aeb-4c79-bbdf-f25ff82a8b89 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:31.234120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9a6b-8aeb-4c79-bbdf-f25ff82a8b89 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:31.234292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9a6b-8aeb-4c79-bbdf-f25ff82a8b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:31.254373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-118c9a6b-8aeb-4c79-bbdf-f25ff82a8b89 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:31.255573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-118c9a6b-8aeb-4c79-bbdf-f25ff82a8b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:31.263334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:31.416734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68f079-a114-46a6-a50a-a4c8b5db8c5e None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:31.431339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68f079-a114-46a6-a50a-a4c8b5db8c5e None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:31.436622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313954c3-a97f-4f41-a8fb-ad380557f220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:31.437241 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313954c3-a97f-4f41-a8fb-ad380557f220 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:31.437518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313954c3-a97f-4f41-a8fb-ad380557f220 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:31.437817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313954c3-a97f-4f41-a8fb-ad380557f220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:31.446226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313954c3-a97f-4f41-a8fb-ad380557f220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:31.447832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313954c3-a97f-4f41-a8fb-ad380557f220 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:31.457174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 721/1437] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:32.260374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a7ac4-969d-40fc-8392-3f8a231fe842 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:32.260979 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a7ac4-969d-40fc-8392-3f8a231fe842 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:32.260979 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a7ac4-969d-40fc-8392-3f8a231fe842 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:32.262238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:32.267666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabf420-3cc9-4f63-8c1d-493d96962d39 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:32.268170 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabf420-3cc9-4f63-8c1d-493d96962d39 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:32.268321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabf420-3cc9-4f63-8c1d-493d96962d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:32.286252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7cabf420-3cc9-4f63-8c1d-493d96962d39 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:32.287455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7cabf420-3cc9-4f63-8c1d-493d96962d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:32.295184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:32.445923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a7ac4-969d-40fc-8392-3f8a231fe842 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:32.460691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a7ac4-969d-40fc-8392-3f8a231fe842 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:32.466041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a7269-e0fa-490b-9a81-cc6e201bf237 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:32.466670 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a7269-e0fa-490b-9a81-cc6e201bf237 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:32.466949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a7269-e0fa-490b-9a81-cc6e201bf237 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:32.467221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a7269-e0fa-490b-9a81-cc6e201bf237 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:32.475391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a7269-e0fa-490b-9a81-cc6e201bf237 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:32.476968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a7269-e0fa-490b-9a81-cc6e201bf237 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:32.486876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 718/1440] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:34.454943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c1355-458b-45ed-8bf5-842ecc06fddd None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:34.455512 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c1355-458b-45ed-8bf5-842ecc06fddd None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:34.455512 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c1355-458b-45ed-8bf5-842ecc06fddd None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:34.457408 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:34.462746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6984c8-dc2a-4d19-9ac2-3cc101938959 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:34.463156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6984c8-dc2a-4d19-9ac2-3cc101938959 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:34.463283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6984c8-dc2a-4d19-9ac2-3cc101938959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:34.481105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6f6984c8-dc2a-4d19-9ac2-3cc101938959 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:34.482331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6f6984c8-dc2a-4d19-9ac2-3cc101938959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:34.490702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:46:34.529128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c1355-458b-45ed-8bf5-842ecc06fddd None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:34.531168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fd2c1355-458b-45ed-8bf5-842ecc06fddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:34.544521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c1355-458b-45ed-8bf5-842ecc06fddd None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:34.548029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d9e5f-a7b8-4d66-a4f9-f3aee1ad92d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:34.549317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d9e5f-a7b8-4d66-a4f9-f3aee1ad92d6 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:34.549620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d9e5f-a7b8-4d66-a4f9-f3aee1ad92d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:34.549900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d9e5f-a7b8-4d66-a4f9-f3aee1ad92d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:34.559002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d9e5f-a7b8-4d66-a4f9-f3aee1ad92d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:34.560423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d9e5f-a7b8-4d66-a4f9-f3aee1ad92d6 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:34.570052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 724/1443] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:36.767749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04faaf21-a49c-4869-9e92-6d3f330b3e30 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:36.768297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04faaf21-a49c-4869-9e92-6d3f330b3e30 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:36.768297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04faaf21-a49c-4869-9e92-6d3f330b3e30 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:36.769697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:36.775135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb16ad7-6c71-4af8-a5dd-2ae3ab46e5eb None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:36.775544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb16ad7-6c71-4af8-a5dd-2ae3ab46e5eb None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:36.775684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb16ad7-6c71-4af8-a5dd-2ae3ab46e5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:36.793753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-dfb16ad7-6c71-4af8-a5dd-2ae3ab46e5eb None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:36.794950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-dfb16ad7-6c71-4af8-a5dd-2ae3ab46e5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:36.803506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:46:36.842124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04faaf21-a49c-4869-9e92-6d3f330b3e30 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:36.852653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04faaf21-a49c-4869-9e92-6d3f330b3e30 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:36.856181 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309245e4-a4bb-4317-8d93-39b1f901ffa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:36.856807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309245e4-a4bb-4317-8d93-39b1f901ffa0 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:36.857089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309245e4-a4bb-4317-8d93-39b1f901ffa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:36.857362 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309245e4-a4bb-4317-8d93-39b1f901ffa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:36.880787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309245e4-a4bb-4317-8d93-39b1f901ffa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:36.882248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309245e4-a4bb-4317-8d93-39b1f901ffa0 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:36.892667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 721/1446] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:39.285663 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df120dbf-3cf4-4543-a497-4d3ed962e0e5 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:39.286424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df120dbf-3cf4-4543-a497-4d3ed962e0e5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:39.286424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df120dbf-3cf4-4543-a497-4d3ed962e0e5 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:39.287490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:39.292646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c3c37-cbc4-4afb-98c9-81f19e3d3318 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:39.293071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c3c37-cbc4-4afb-98c9-81f19e3d3318 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:39.293189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c3c37-cbc4-4afb-98c9-81f19e3d3318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:39.311687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d48c3c37-cbc4-4afb-98c9-81f19e3d3318 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:39.313048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d48c3c37-cbc4-4afb-98c9-81f19e3d3318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:39.322819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:39.363200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df120dbf-3cf4-4543-a497-4d3ed962e0e5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:39.373785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df120dbf-3cf4-4543-a497-4d3ed962e0e5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:39.377929 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f324dc6-1297-4b6c-9529-9a758bbcde1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:39.378567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f324dc6-1297-4b6c-9529-9a758bbcde1b service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:39.378849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f324dc6-1297-4b6c-9529-9a758bbcde1b service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:39.379126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f324dc6-1297-4b6c-9529-9a758bbcde1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:39.387888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f324dc6-1297-4b6c-9529-9a758bbcde1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:39.389328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f324dc6-1297-4b6c-9529-9a758bbcde1b service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:39.398749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 727/1449] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:41.572456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3407b-9542-4585-85a7-d8f0f63a6f7c None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:41.572907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3407b-9542-4585-85a7-d8f0f63a6f7c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:41.573020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3407b-9542-4585-85a7-d8f0f63a6f7c None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:41.574414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:41.580058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb33b40-2c73-424b-acc6-ab529de41f90 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:41.580530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb33b40-2c73-424b-acc6-ab529de41f90 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:41.580696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb33b40-2c73-424b-acc6-ab529de41f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:41.598959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-4fb33b40-2c73-424b-acc6-ab529de41f90 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:41.600153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4fb33b40-2c73-424b-acc6-ab529de41f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:41.607971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:41.646859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3407b-9542-4585-85a7-d8f0f63a6f7c None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:41.658197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3407b-9542-4585-85a7-d8f0f63a6f7c None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:41.661960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d78c1-92ba-4a8c-8a7e-d467cf5db005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:41.662582 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d78c1-92ba-4a8c-8a7e-d467cf5db005 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:41.662857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d78c1-92ba-4a8c-8a7e-d467cf5db005 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:41.663126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d78c1-92ba-4a8c-8a7e-d467cf5db005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:41.671762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d78c1-92ba-4a8c-8a7e-d467cf5db005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:41.673604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d78c1-92ba-4a8c-8a7e-d467cf5db005 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:41.682687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 724/1452] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:43.842959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bfbcc-7748-43d7-ab1f-766f631fb0bf None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:43.843508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bfbcc-7748-43d7-ab1f-766f631fb0bf None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:43.843508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bfbcc-7748-43d7-ab1f-766f631fb0bf None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:43.844900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:43.850883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b375-9cbc-4da5-bd56-2795c0c9e7ef None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:43.851344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b375-9cbc-4da5-bd56-2795c0c9e7ef None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:43.851442 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51b375-9cbc-4da5-bd56-2795c0c9e7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:43.869281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5b51b375-9cbc-4da5-bd56-2795c0c9e7ef None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:43.870531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5b51b375-9cbc-4da5-bd56-2795c0c9e7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:43.879459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:43.916147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bfbcc-7748-43d7-ab1f-766f631fb0bf None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:43.927600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bfbcc-7748-43d7-ab1f-766f631fb0bf None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:43.931116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5e0ec-7856-4100-a055-87a6fd827551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:43.931739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5e0ec-7856-4100-a055-87a6fd827551 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:43.932013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5e0ec-7856-4100-a055-87a6fd827551 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:43.932287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5e0ec-7856-4100-a055-87a6fd827551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:43.941765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5e0ec-7856-4100-a055-87a6fd827551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:43.943188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5e0ec-7856-4100-a055-87a6fd827551 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:43.952532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 730/1455] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:46.065377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3848f-ae7d-42f6-abe4-fcb2962653f2 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:46.065943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3848f-ae7d-42f6-abe4-fcb2962653f2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:46.065943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3848f-ae7d-42f6-abe4-fcb2962653f2 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:46.067342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:46.072582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c72be-ca10-49c4-b5f6-72013f4cf1c5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:46.073020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c72be-ca10-49c4-b5f6-72013f4cf1c5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:46.073129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c72be-ca10-49c4-b5f6-72013f4cf1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:46.091664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-839c72be-ca10-49c4-b5f6-72013f4cf1c5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:46.092856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-839c72be-ca10-49c4-b5f6-72013f4cf1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:46.101179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:46.138147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3848f-ae7d-42f6-abe4-fcb2962653f2 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:46.149312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3848f-ae7d-42f6-abe4-fcb2962653f2 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:46.152772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83256f5-628b-454c-b44f-798d74deb151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:46.153425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83256f5-628b-454c-b44f-798d74deb151 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:46.153736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83256f5-628b-454c-b44f-798d74deb151 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:46.154010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83256f5-628b-454c-b44f-798d74deb151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:46.163647 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83256f5-628b-454c-b44f-798d74deb151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:46.165100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83256f5-628b-454c-b44f-798d74deb151 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:46.174423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 727/1458] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:48.316941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26e0e-7195-4caf-9025-2422438c7a2b None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:48.317534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26e0e-7195-4caf-9025-2422438c7a2b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:48.317534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26e0e-7195-4caf-9025-2422438c7a2b None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:48.318939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:48.324030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390c5f-a585-4b35-8306-97fa81c8bf6a None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:48.324426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390c5f-a585-4b35-8306-97fa81c8bf6a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:48.324577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390c5f-a585-4b35-8306-97fa81c8bf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:48.342264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-8d390c5f-a585-4b35-8306-97fa81c8bf6a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:48.343446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8d390c5f-a585-4b35-8306-97fa81c8bf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:48.351596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:48.391138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af26e0e-7195-4caf-9025-2422438c7a2b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:48.401888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af26e0e-7195-4caf-9025-2422438c7a2b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:48.405656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac9078-5e55-482c-98d0-25631060a458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:48.406287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac9078-5e55-482c-98d0-25631060a458 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:48.406568 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac9078-5e55-482c-98d0-25631060a458 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:48.406841 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac9078-5e55-482c-98d0-25631060a458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:48.415678 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac9078-5e55-482c-98d0-25631060a458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:48.417091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac9078-5e55-482c-98d0-25631060a458 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:48.426874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 733/1461] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:50.518945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b233b18-f6a5-4268-a93d-1d3c85658259 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:50.519498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b233b18-f6a5-4268-a93d-1d3c85658259 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:50.519498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b233b18-f6a5-4268-a93d-1d3c85658259 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:50.520872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:50.526402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40327da-b7ec-473e-98d9-aa137d79b9c5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:50.526830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40327da-b7ec-473e-98d9-aa137d79b9c5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:50.526969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40327da-b7ec-473e-98d9-aa137d79b9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:50.533200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=79 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:46:50.545795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e40327da-b7ec-473e-98d9-aa137d79b9c5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:50.547003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e40327da-b7ec-473e-98d9-aa137d79b9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:50.555500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:50.593243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b233b18-f6a5-4268-a93d-1d3c85658259 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:50.598847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=73 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:46:50.605129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b233b18-f6a5-4268-a93d-1d3c85658259 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:50.608574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87509f37-e205-4479-bd48-4939a82eca42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:50.609203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87509f37-e205-4479-bd48-4939a82eca42 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:50.609482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87509f37-e205-4479-bd48-4939a82eca42 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:50.609789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87509f37-e205-4479-bd48-4939a82eca42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:50.619245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87509f37-e205-4479-bd48-4939a82eca42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:50.620699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87509f37-e205-4479-bd48-4939a82eca42 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:50.630787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 730/1464] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:52.692156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb175e8-0878-496c-b05b-64a3f6b7409c None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:52.692705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb175e8-0878-496c-b05b-64a3f6b7409c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:52.692705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb175e8-0878-496c-b05b-64a3f6b7409c None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:52.694134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:52.699380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938e5f1-7247-4bbd-8479-a5cd2718b531 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:52.699818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938e5f1-7247-4bbd-8479-a5cd2718b531 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:52.699930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938e5f1-7247-4bbd-8479-a5cd2718b531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:52.717825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a938e5f1-7247-4bbd-8479-a5cd2718b531 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:52.719023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a938e5f1-7247-4bbd-8479-a5cd2718b531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:52.727702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:52.764392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb175e8-0878-496c-b05b-64a3f6b7409c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:52.781355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb175e8-0878-496c-b05b-64a3f6b7409c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:52.786060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8eb46a-ce67-41ac-abf1-bc9b267d4751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:52.786692 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8eb46a-ce67-41ac-abf1-bc9b267d4751 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:52.786970 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8eb46a-ce67-41ac-abf1-bc9b267d4751 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:52.787239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8eb46a-ce67-41ac-abf1-bc9b267d4751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:52.796377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8eb46a-ce67-41ac-abf1-bc9b267d4751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:52.797842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8eb46a-ce67-41ac-abf1-bc9b267d4751 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:52.807637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 736/1467] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:46:54.884225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd43c2-d7cf-4c05-8e86-70511af9577f None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:54.884761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd43c2-d7cf-4c05-8e86-70511af9577f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:54.884761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd43c2-d7cf-4c05-8e86-70511af9577f None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:54.886152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:46:54.891693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb67ad7-7ab1-4573-8b97-f797364b8a17 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:54.892623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb67ad7-7ab1-4573-8b97-f797364b8a17 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:54.892720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb67ad7-7ab1-4573-8b97-f797364b8a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:54.910254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-6cb67ad7-7ab1-4573-8b97-f797364b8a17 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:46:54.911454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6cb67ad7-7ab1-4573-8b97-f797364b8a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:46:54.922210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:46:54.959429 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd43c2-d7cf-4c05-8e86-70511af9577f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:46:54.970400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd43c2-d7cf-4c05-8e86-70511af9577f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:46:54.973928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073b92d-e01e-4a4d-b05f-ac9ae50338ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:46:54.974580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073b92d-e01e-4a4d-b05f-ac9ae50338ff service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:46:54.974859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073b92d-e01e-4a4d-b05f-ac9ae50338ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:46:54.975132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073b92d-e01e-4a4d-b05f-ac9ae50338ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:46:54.983920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073b92d-e01e-4a4d-b05f-ac9ae50338ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:46:54.985866 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073b92d-e01e-4a4d-b05f-ac9ae50338ff service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:46:54.995078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 733/1470] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:46:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:47:04.905340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=11 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:47:04.990973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=14 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:47:15.085283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a9656-25c0-457e-bec0-9963a0118351 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:15.085871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a9656-25c0-457e-bec0-9963a0118351 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:15.085871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a9656-25c0-457e-bec0-9963a0118351 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:15.087263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 738/1471] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:47:15.092648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6296ae7-988e-4b58-aa83-a7fd404225ed None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:15.093042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6296ae7-988e-4b58-aa83-a7fd404225ed None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:15.093208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6296ae7-988e-4b58-aa83-a7fd404225ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:15.111842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-e6296ae7-988e-4b58-aa83-a7fd404225ed None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:47:15.113030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e6296ae7-988e-4b58-aa83-a7fd404225ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:47:15.120721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 734/1472] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:47:15.167934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b151a-d8c8-4a3c-8129-c4d9b2be7651 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:15.168226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b151a-d8c8-4a3c-8129-c4d9b2be7651 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:15.168505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b151a-d8c8-4a3c-8129-c4d9b2be7651 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:15.169924 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 739/1473] 10.222.0.66 () {54 vars in 962 bytes} [Fri Jul 1 19:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:47:15.175333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d0b0a-b5d6-4fa7-8893-d93e9ed93573 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:15.175618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d0b0a-b5d6-4fa7-8893-d93e9ed93573 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:15.175890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d0b0a-b5d6-4fa7-8893-d93e9ed93573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:15.194083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-3a7d0b0a-b5d6-4fa7-8893-d93e9ed93573 None None] MFA Rules not processed for user `c022224b03a54ab489d95502c63a25dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:47:15.195340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3a7d0b0a-b5d6-4fa7-8893-d93e9ed93573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:47:15.201654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 735/1474] 10.222.0.66 () {58 vars in 1059 bytes} [Fri Jul 1 19:47:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:47:15.208859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b151a-d8c8-4a3c-8129-c4d9b2be7651 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:47:15.219916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b151a-d8c8-4a3c-8129-c4d9b2be7651 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:47:15.223419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff0eaf-7f5c-4b36-b5d5-8f273aedb55f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c022224b03a54ab489d95502c63a25dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:47:15.224486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff0eaf-7f5c-4b36-b5d5-8f273aedb55f service glance] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:15.225397 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff0eaf-7f5c-4b36-b5d5-8f273aedb55f service glance] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:15.225915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff0eaf-7f5c-4b36-b5d5-8f273aedb55f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:15.234734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff0eaf-7f5c-4b36-b5d5-8f273aedb55f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:47:15.236749 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff0eaf-7f5c-4b36-b5d5-8f273aedb55f service glance] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:47:15.246095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 740/1475] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:47:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:47:17.430703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7032e92-8082-4dcf-b1bf-d35f650b5427 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:17.431272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7032e92-8082-4dcf-b1bf-d35f650b5427 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:17.431272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7032e92-8082-4dcf-b1bf-d35f650b5427 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:17.432614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 736/1476] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:47:17.437864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7245be-1777-4998-87be-400b31772a1b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:17.439269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7245be-1777-4998-87be-400b31772a1b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:17.439269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7245be-1777-4998-87be-400b31772a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:17.456876 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ba7245be-1777-4998-87be-400b31772a1b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:47:17.458102 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ba7245be-1777-4998-87be-400b31772a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:47:17.466109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 741/1477] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:47:17.489912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7032e92-8082-4dcf-b1bf-d35f650b5427 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:47:17.503549 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7032e92-8082-4dcf-b1bf-d35f650b5427 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:47:17.507978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339228a6-f264-4bc8-97f2-4027af00493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:47:17.508627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339228a6-f264-4bc8-97f2-4027af00493f admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:17.508908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339228a6-f264-4bc8-97f2-4027af00493f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:17.509181 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339228a6-f264-4bc8-97f2-4027af00493f admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:17.509949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339228a6-f264-4bc8-97f2-4027af00493f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:47:17.511726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: WARNING py.warnings [None req-339228a6-f264-4bc8-97f2-4027af00493f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:47:17.511726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: warnings.warn(msg) Jul 01 19:47:17.511726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: Jul 01 19:47:17.512872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339228a6-f264-4bc8-97f2-4027af00493f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:47:17.531987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 737/1478] 10.222.0.66 () {56 vars in 1108 bytes} [Fri Jul 1 19:47:17 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:47:19.656457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d52fb-6771-4f9c-9d59-91bd7dacf12a None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:19.657499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d52fb-6771-4f9c-9d59-91bd7dacf12a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:19.657499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d52fb-6771-4f9c-9d59-91bd7dacf12a None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:19.658897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 742/1479] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:47:19.664173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90d090-9fbf-4c5a-a237-0f21c279be03 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:19.664613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90d090-9fbf-4c5a-a237-0f21c279be03 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:19.664753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90d090-9fbf-4c5a-a237-0f21c279be03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:19.683305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9f90d090-9fbf-4c5a-a237-0f21c279be03 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:47:19.684542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9f90d090-9fbf-4c5a-a237-0f21c279be03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:47:19.692606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 738/1480] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:47:19.736804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d52fb-6771-4f9c-9d59-91bd7dacf12a None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:47:19.751281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d52fb-6771-4f9c-9d59-91bd7dacf12a None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:47:19.756933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349fd1e-ee8d-4c19-9840-a277fcea3095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:47:19.757951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349fd1e-ee8d-4c19-9840-a277fcea3095 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:19.758234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349fd1e-ee8d-4c19-9840-a277fcea3095 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:19.758505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349fd1e-ee8d-4c19-9840-a277fcea3095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:19.767039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349fd1e-ee8d-4c19-9840-a277fcea3095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:47:19.768626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349fd1e-ee8d-4c19-9840-a277fcea3095 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:47:19.777868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 743/1481] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:47:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:47:22.033050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c7661-41b4-4ca7-94ae-fc7940078b5e None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:22.033637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c7661-41b4-4ca7-94ae-fc7940078b5e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:22.034049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c7661-41b4-4ca7-94ae-fc7940078b5e None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:22.036547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 739/1482] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:47:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:47:22.041708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94a731-f2c8-4aec-a524-e93c8c4e8687 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:22.042130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94a731-f2c8-4aec-a524-e93c8c4e8687 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:22.042264 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94a731-f2c8-4aec-a524-e93c8c4e8687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:22.060526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9b94a731-f2c8-4aec-a524-e93c8c4e8687 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:47:22.061762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9b94a731-f2c8-4aec-a524-e93c8c4e8687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:47:22.069825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 744/1483] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:47:22.114822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c7661-41b4-4ca7-94ae-fc7940078b5e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:47:22.127459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c7661-41b4-4ca7-94ae-fc7940078b5e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:47:22.131209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e38f0d-bf71-4c58-b43a-b1e54537c300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:47:22.131863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e38f0d-bf71-4c58-b43a-b1e54537c300 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:22.132142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e38f0d-bf71-4c58-b43a-b1e54537c300 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:22.132420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e38f0d-bf71-4c58-b43a-b1e54537c300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:22.143033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e38f0d-bf71-4c58-b43a-b1e54537c300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:47:22.144488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e38f0d-bf71-4c58-b43a-b1e54537c300 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:47:22.155784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 740/1484] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:47:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:47:24.265176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4b42-b352-4eed-b627-e2f0ca57421e None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:24.265736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4b42-b352-4eed-b627-e2f0ca57421e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:24.265736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4b42-b352-4eed-b627-e2f0ca57421e None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:24.267186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 745/1485] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:47:24.272606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eea229-e68e-4925-b0e3-331bbf20cfc3 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:24.273073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eea229-e68e-4925-b0e3-331bbf20cfc3 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:24.273169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eea229-e68e-4925-b0e3-331bbf20cfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:24.290926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-73eea229-e68e-4925-b0e3-331bbf20cfc3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:47:24.292122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-73eea229-e68e-4925-b0e3-331bbf20cfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:47:24.300479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 741/1486] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:47:24.339095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4b42-b352-4eed-b627-e2f0ca57421e None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:47:24.349945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4b42-b352-4eed-b627-e2f0ca57421e None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:47:24.353377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badc605-b6ca-4068-9231-46de70e8ac60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:47:24.354037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badc605-b6ca-4068-9231-46de70e8ac60 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:24.354318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badc605-b6ca-4068-9231-46de70e8ac60 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:24.354590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badc605-b6ca-4068-9231-46de70e8ac60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:24.363787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badc605-b6ca-4068-9231-46de70e8ac60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:47:24.365248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badc605-b6ca-4068-9231-46de70e8ac60 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:47:24.374828 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 746/1487] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:47:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:47:26.635410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ee0a2-95fc-42cc-9d62-cb599defae4f None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:26.635937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ee0a2-95fc-42cc-9d62-cb599defae4f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:26.635937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ee0a2-95fc-42cc-9d62-cb599defae4f None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:26.637365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 742/1488] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:47:26.643068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39170b-af62-4035-87d0-b9e6fb18aa9b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:26.643502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39170b-af62-4035-87d0-b9e6fb18aa9b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:26.643623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39170b-af62-4035-87d0-b9e6fb18aa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:26.662004 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-1a39170b-af62-4035-87d0-b9e6fb18aa9b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:47:26.663203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1a39170b-af62-4035-87d0-b9e6fb18aa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:47:26.670997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 747/1489] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:47:26.711215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ee0a2-95fc-42cc-9d62-cb599defae4f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:47:26.722935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ee0a2-95fc-42cc-9d62-cb599defae4f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:47:26.726463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf208510-7c52-46f8-91bc-629678ed2f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:47:26.727099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf208510-7c52-46f8-91bc-629678ed2f64 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:26.727382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf208510-7c52-46f8-91bc-629678ed2f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:26.727658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf208510-7c52-46f8-91bc-629678ed2f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:26.737377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf208510-7c52-46f8-91bc-629678ed2f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:47:26.739035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf208510-7c52-46f8-91bc-629678ed2f64 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:47:26.748819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 743/1490] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:47:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:47:28.945165 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49f0d8-514d-4748-8562-ec11f833171e None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:28.945779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49f0d8-514d-4748-8562-ec11f833171e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:28.945779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49f0d8-514d-4748-8562-ec11f833171e None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:28.947106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 748/1491] 10.222.0.66 () {54 vars in 926 bytes} [Fri Jul 1 19:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:47:28.952203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66160b6-b77c-4f24-8443-9689bf4de8d2 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:28.952630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66160b6-b77c-4f24-8443-9689bf4de8d2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:28.952745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66160b6-b77c-4f24-8443-9689bf4de8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:28.971714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d66160b6-b77c-4f24-8443-9689bf4de8d2 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:47:28.972910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d66160b6-b77c-4f24-8443-9689bf4de8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:47:28.980640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 744/1492] 10.222.0.66 () {58 vars in 1023 bytes} [Fri Jul 1 19:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:47:29.019007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49f0d8-514d-4748-8562-ec11f833171e None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:47:29.029418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49f0d8-514d-4748-8562-ec11f833171e None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:47:29.033677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8d208-437c-4768-aa4e-5525fa11f011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:47:29.034379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8d208-437c-4768-aa4e-5525fa11f011 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:47:29.034656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8d208-437c-4768-aa4e-5525fa11f011 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:47:29.034926 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8d208-437c-4768-aa4e-5525fa11f011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:47:29.043580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8d208-437c-4768-aa4e-5525fa11f011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:47:29.044989 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8d208-437c-4768-aa4e-5525fa11f011 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:47:29.054727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 749/1493] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:47:38.966072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=33 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:47:39.050902 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=33 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:48:57.362050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc148fc0-ac49-4588-b429-b9d96f80f356 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.362605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc148fc0-ac49-4588-b429-b9d96f80f356 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.362605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc148fc0-ac49-4588-b429-b9d96f80f356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.384799 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-dc148fc0-ac49-4588-b429-b9d96f80f356 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:48:57.386285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-dc148fc0-ac49-4588-b429-b9d96f80f356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:48:57.394162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 745/1494] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:48:57.402213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed5bc-dd31-4693-b86c-8b6c5996a9b6 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.402698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed5bc-dd31-4693-b86c-8b6c5996a9b6 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.402802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed5bc-dd31-4693-b86c-8b6c5996a9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.421792 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-189ed5bc-dd31-4693-b86c-8b6c5996a9b6 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:48:57.422985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-189ed5bc-dd31-4693-b86c-8b6c5996a9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:48:57.430690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 750/1495] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:48:57.435116 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc148fc0-ac49-4588-b429-b9d96f80f356 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:48:57.445266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc148fc0-ac49-4588-b429-b9d96f80f356 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:48:57.449710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb88cb0-4ce9-48ca-bc37-f7a6be9cdd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:48:57.450360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb88cb0-4ce9-48ca-bc37-f7a6be9cdd4e admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.450638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb88cb0-4ce9-48ca-bc37-f7a6be9cdd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.450909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb88cb0-4ce9-48ca-bc37-f7a6be9cdd4e admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.451625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb88cb0-4ce9-48ca-bc37-f7a6be9cdd4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:48:57.451926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb88cb0-4ce9-48ca-bc37-f7a6be9cdd4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:48:57.453024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: WARNING py.warnings [None req-2bb88cb0-4ce9-48ca-bc37-f7a6be9cdd4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:48:57.453024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: warnings.warn(msg) Jul 01 19:48:57.453024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: Jul 01 19:48:57.453944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb88cb0-4ce9-48ca-bc37-f7a6be9cdd4e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:48:57.465233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 746/1496] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:48:57.473400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ed5bc-dd31-4693-b86c-8b6c5996a9b6 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:48:57.482556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ed5bc-dd31-4693-b86c-8b6c5996a9b6 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:48:57.487579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8d1bf-e458-4cd6-9044-c99a5bef7ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:48:57.488218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8d1bf-e458-4cd6-9044-c99a5bef7ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.488503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8d1bf-e458-4cd6-9044-c99a5bef7ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.488780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8d1bf-e458-4cd6-9044-c99a5bef7ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.489474 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8d1bf-e458-4cd6-9044-c99a5bef7ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:48:57.491277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: WARNING py.warnings [None req-f2f8d1bf-e458-4cd6-9044-c99a5bef7ef3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:48:57.491277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: warnings.warn(msg) Jul 01 19:48:57.491277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: Jul 01 19:48:57.492469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8d1bf-e458-4cd6-9044-c99a5bef7ef3 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:48:57.529024 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 751/1497] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:48:57 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:48:57.533516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb88cb0-4ce9-48ca-bc37-f7a6be9cdd4e admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:48:57.543509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb88cb0-4ce9-48ca-bc37-f7a6be9cdd4e admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:48:57.599956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6a700-9df8-4e9a-9905-0c74dfe13ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:48:57.600628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6a700-9df8-4e9a-9905-0c74dfe13ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.600910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6a700-9df8-4e9a-9905-0c74dfe13ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.601183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6a700-9df8-4e9a-9905-0c74dfe13ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.601912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6a700-9df8-4e9a-9905-0c74dfe13ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:48:57.602965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: WARNING py.warnings [None req-39d6a700-9df8-4e9a-9905-0c74dfe13ff8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:48:57.602965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: warnings.warn(msg) Jul 01 19:48:57.602965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: Jul 01 19:48:57.604162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6a700-9df8-4e9a-9905-0c74dfe13ff8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:48:57.624115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 747/1498] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:48:57 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:48:57.628169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8d1bf-e458-4cd6-9044-c99a5bef7ef3 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:48:57.638229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8d1bf-e458-4cd6-9044-c99a5bef7ef3 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:48:57.643160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dd323-b09e-4531-81f8-7e05b0b7043a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:48:57.644121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dd323-b09e-4531-81f8-7e05b0b7043a admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.644544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dd323-b09e-4531-81f8-7e05b0b7043a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.644952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dd323-b09e-4531-81f8-7e05b0b7043a admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.646128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56dd323-b09e-4531-81f8-7e05b0b7043a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:48:57.647826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: WARNING py.warnings [None req-e56dd323-b09e-4531-81f8-7e05b0b7043a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:48:57.647826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: warnings.warn(msg) Jul 01 19:48:57.647826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: Jul 01 19:48:57.648724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56dd323-b09e-4531-81f8-7e05b0b7043a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:48:57.662837 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 752/1499] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:48:57.666810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6a700-9df8-4e9a-9905-0c74dfe13ff8 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:48:57.678246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6a700-9df8-4e9a-9905-0c74dfe13ff8 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:48:57.682755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ee41a-8929-41bf-99ba-057e7432cd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:48:57.683409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ee41a-8929-41bf-99ba-057e7432cd1f admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.683691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ee41a-8929-41bf-99ba-057e7432cd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.683968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ee41a-8929-41bf-99ba-057e7432cd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.684690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ee41a-8929-41bf-99ba-057e7432cd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:48:57.685739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: WARNING py.warnings [None req-b74ee41a-8929-41bf-99ba-057e7432cd1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:48:57.685739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: warnings.warn(msg) Jul 01 19:48:57.685739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: Jul 01 19:48:57.686617 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ee41a-8929-41bf-99ba-057e7432cd1f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:48:57.699770 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 748/1500] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:48:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:48:57.703844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dd323-b09e-4531-81f8-7e05b0b7043a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:48:57.713724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dd323-b09e-4531-81f8-7e05b0b7043a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:48:57.718734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1c7f1-03aa-459e-a59c-1c7b928519e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:48:57.719610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1c7f1-03aa-459e-a59c-1c7b928519e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.719892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1c7f1-03aa-459e-a59c-1c7b928519e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.720170 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1c7f1-03aa-459e-a59c-1c7b928519e8 admin admin] PATH_INFO: `/v3/projects/02b806d1bbc942919d2a53851a861d8a/users/022b369c168843308bdfc9b1e585a256/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.735018 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1c7f1-03aa-459e-a59c-1c7b928519e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b806d1bbc942919d2a53851a861d8a, user_id=022b369c168843308bdfc9b1e585a256, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:48:57.736040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: WARNING py.warnings [None req-6db1c7f1-03aa-459e-a59c-1c7b928519e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:48:57.736040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: warnings.warn(msg) Jul 01 19:48:57.736040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: Jul 01 19:48:57.738002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1c7f1-03aa-459e-a59c-1c7b928519e8 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:48:57.750781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 753/1501] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:48:57 2022] PUT /identity/v3/projects/02b806d1bbc942919d2a53851a861d8a/users/022b369c168843308bdfc9b1e585a256/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:48:57.762709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb9cce-30dc-4139-9291-132ce64d6198 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.762998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb9cce-30dc-4139-9291-132ce64d6198 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.763300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb9cce-30dc-4139-9291-132ce64d6198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.789513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-3bcb9cce-30dc-4139-9291-132ce64d6198 None None] MFA Rules not processed for user `022b369c168843308bdfc9b1e585a256`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:48:57.790736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3bcb9cce-30dc-4139-9291-132ce64d6198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:48:57.853692 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 749/1502] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:48:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:48:57.867181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57830b-dce7-47c8-bb85-6bb9cf2a4ff8 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.867488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57830b-dce7-47c8-bb85-6bb9cf2a4ff8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.867764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57830b-dce7-47c8-bb85-6bb9cf2a4ff8 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.869158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 754/1503] 10.222.0.66 () {54 vars in 963 bytes} [Fri Jul 1 19:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:48:57.874272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39a53-79e8-4a19-8455-51e57defb220 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.874625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39a53-79e8-4a19-8455-51e57defb220 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.874880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39a53-79e8-4a19-8455-51e57defb220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.892535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ccb39a53-79e8-4a19-8455-51e57defb220 None None] MFA Rules not processed for user `a4b0e6ea57af454cbc8f5bd180bc052e`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:48:57.893838 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ccb39a53-79e8-4a19-8455-51e57defb220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:48:57.936100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:48:57.944819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 750/1504] 10.222.0.66 () {58 vars in 1060 bytes} [Fri Jul 1 19:48:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:48:57.953632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57830b-dce7-47c8-bb85-6bb9cf2a4ff8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:48:57.964571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57830b-dce7-47c8-bb85-6bb9cf2a4ff8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:48:57.968672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0c3ef-1429-4f95-b375-4fecfd082cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:48:57.969474 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0c3ef-1429-4f95-b375-4fecfd082cca service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:57.969787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0c3ef-1429-4f95-b375-4fecfd082cca service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:57.970057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0c3ef-1429-4f95-b375-4fecfd082cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:57.978234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0c3ef-1429-4f95-b375-4fecfd082cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:48:57.979645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0c3ef-1429-4f95-b375-4fecfd082cca service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:48:57.988259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 755/1505] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:48:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:48:58.029240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9a11e-0e63-472a-8b10-b79ace4b1625 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:58.029545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9a11e-0e63-472a-8b10-b79ace4b1625 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:58.029844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9a11e-0e63-472a-8b10-b79ace4b1625 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:58.032223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 751/1506] 10.222.0.66 () {52 vars in 829 bytes} [Fri Jul 1 19:48:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:48:58.035687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0c3ef-1429-4f95-b375-4fecfd082cca service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:48:58.099602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0c3ef-1429-4f95-b375-4fecfd082cca service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:48:58.104044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:48:58.104907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:58.105186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:58.105455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] PATH_INFO: `/v3/users/022b369c168843308bdfc9b1e585a256` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:58.106654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=022b369c168843308bdfc9b1e585a256)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:48:58.107666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: WARNING py.warnings [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:48:58.107666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: warnings.warn(msg) Jul 01 19:48:58.107666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: Jul 01 19:48:58.109045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:48:58.172317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '022b369c168843308bdfc9b1e585a256'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:48:58.190802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '022b369c168843308bdfc9b1e585a256'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:48:58.200240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '022b369c168843308bdfc9b1e585a256'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:48:58.211900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:48:58.217415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '022b369c168843308bdfc9b1e585a256'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:48:58.219399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '022b369c168843308bdfc9b1e585a256'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:48:58.227433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 756/1507] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:48:58 2022] DELETE /identity/v3/users/022b369c168843308bdfc9b1e585a256 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:48:58.232208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9fa31-0133-4864-a035-c31d326b8523 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:58.232498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9fa31-0133-4864-a035-c31d326b8523 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:58.232778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9fa31-0133-4864-a035-c31d326b8523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:58.249956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-89a9fa31-0133-4864-a035-c31d326b8523 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:48:58.251155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-89a9fa31-0133-4864-a035-c31d326b8523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:48:58.311334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 752/1508] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:48:58.321872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:48:58.323310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:48:58.379090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cdbb4-ed72-4796-a234-afa8b7951608 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:48:58.382642 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5b38f-7212-4e4d-af3a-0cae3e54fe61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:48:58.383280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5b38f-7212-4e4d-af3a-0cae3e54fe61 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:58.383562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5b38f-7212-4e4d-af3a-0cae3e54fe61 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:58.383836 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5b38f-7212-4e4d-af3a-0cae3e54fe61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:58.393310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5b38f-7212-4e4d-af3a-0cae3e54fe61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:48:58.394771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5b38f-7212-4e4d-af3a-0cae3e54fe61 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:48:58.404036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 757/1509] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:48:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:48:58.735668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9fa31-0133-4864-a035-c31d326b8523 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:48:58.737155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-89a9fa31-0133-4864-a035-c31d326b8523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:48:58.749095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9fa31-0133-4864-a035-c31d326b8523 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:48:58.754908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:48:58.755800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:48:58.756078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:48:58.756449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] PATH_INFO: `/v3/projects/02b806d1bbc942919d2a53851a861d8a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:48:58.757698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] RBAC: Authorizing `identity:delete_project(project_id=02b806d1bbc942919d2a53851a861d8a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:48:58.758683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: WARNING py.warnings [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:48:58.758683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: warnings.warn(msg) Jul 01 19:48:58.758683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: Jul 01 19:48:58.759874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:48:58.820064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02b806d1bbc942919d2a53851a861d8a', 'request_id': 'req-d3644f11-407a-4de7-ba89-51c51539407c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:48:58.830285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02b806d1bbc942919d2a53851a861d8a', 'request_id': 'req-d3644f11-407a-4de7-ba89-51c51539407c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:48:58.843999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02b806d1bbc942919d2a53851a861d8a', 'request_id': 'req-d3644f11-407a-4de7-ba89-51c51539407c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:48:58.854509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] The token cache is being invalidate because project 02b806d1bbc942919d2a53851a861d8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:48:58.854792 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:48:58.855427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d3644f11-407a-4de7-ba89-51c51539407c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:48:58.857040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 753/1510] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:48:58 2022] DELETE /identity/v3/projects/02b806d1bbc942919d2a53851a861d8a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:49:08.399911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=29 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:49:08.851554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:50:38.996448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74914fc-e984-41c7-9857-b32b73f5647e None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:38.998044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74914fc-e984-41c7-9857-b32b73f5647e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:38.998044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74914fc-e984-41c7-9857-b32b73f5647e None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:38.999006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:50:39.003819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39dbb9-a18d-4e41-a37d-7a0237029e12 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:39.004275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39dbb9-a18d-4e41-a37d-7a0237029e12 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:39.004380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39dbb9-a18d-4e41-a37d-7a0237029e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:39.024457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-df39dbb9-a18d-4e41-a37d-7a0237029e12 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:50:39.025662 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-df39dbb9-a18d-4e41-a37d-7a0237029e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:50:39.088778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:50:39.249451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74914fc-e984-41c7-9857-b32b73f5647e None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:50:39.251144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d74914fc-e984-41c7-9857-b32b73f5647e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:50:39.327854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74914fc-e984-41c7-9857-b32b73f5647e None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:50:39.333128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56139a8e-c6df-42b9-a6b8-c382ef21c9a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:50:39.333788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56139a8e-c6df-42b9-a6b8-c382ef21c9a4 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:39.334066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56139a8e-c6df-42b9-a6b8-c382ef21c9a4 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:39.334334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56139a8e-c6df-42b9-a6b8-c382ef21c9a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:39.343189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56139a8e-c6df-42b9-a6b8-c382ef21c9a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:50:39.344828 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56139a8e-c6df-42b9-a6b8-c382ef21c9a4 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:50:39.354432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 759/1513] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:50:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:50:40.191134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed240dd4-a008-4383-8700-9c6a097cf9b9 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:40.191701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed240dd4-a008-4383-8700-9c6a097cf9b9 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:40.191701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed240dd4-a008-4383-8700-9c6a097cf9b9 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:40.193130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:50:40.197991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0160ca5-2b46-46e3-8108-676bbe843d56 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:40.198412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0160ca5-2b46-46e3-8108-676bbe843d56 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:40.198516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0160ca5-2b46-46e3-8108-676bbe843d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:40.217309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f0160ca5-2b46-46e3-8108-676bbe843d56 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:50:40.218555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f0160ca5-2b46-46e3-8108-676bbe843d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:50:40.226242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:50:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:50:40.379879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed240dd4-a008-4383-8700-9c6a097cf9b9 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:50:40.394036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed240dd4-a008-4383-8700-9c6a097cf9b9 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:50:40.399372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade67a4-95b9-47db-ae8b-75193358a756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:50:40.400004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade67a4-95b9-47db-ae8b-75193358a756 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:40.400283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade67a4-95b9-47db-ae8b-75193358a756 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:40.400555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade67a4-95b9-47db-ae8b-75193358a756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:40.409767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade67a4-95b9-47db-ae8b-75193358a756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:50:40.411344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade67a4-95b9-47db-ae8b-75193358a756 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:50:40.421304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 756/1516] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:50:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:50:41.297883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481685ae-d6e8-4f0a-a2f1-c968df192807 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:41.298400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481685ae-d6e8-4f0a-a2f1-c968df192807 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:41.298400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481685ae-d6e8-4f0a-a2f1-c968df192807 None None] PATH_INFO: `/v3` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:41.299672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 761/1517] 10.222.0.60 () {54 vars in 925 bytes} [Fri Jul 1 19:50:41 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:50:41.305978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3e62f-2633-4714-9fce-a10521e77129 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:41.306420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3e62f-2633-4714-9fce-a10521e77129 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:41.306532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3e62f-2633-4714-9fce-a10521e77129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:41.325482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-b5e3e62f-2633-4714-9fce-a10521e77129 None None] MFA Rules not processed for user `bba7d9205e694f2182d6091061d1219c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:50:41.326618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b5e3e62f-2633-4714-9fce-a10521e77129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:50:41.393302 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 757/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Fri Jul 1 19:50:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:50:45.217469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28cee3-4a5e-4861-afca-a023de559d6c None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:45.218021 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28cee3-4a5e-4861-afca-a023de559d6c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:45.218133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28cee3-4a5e-4861-afca-a023de559d6c None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:45.219419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 762/1519] 10.222.0.66 () {54 vars in 960 bytes} [Fri Jul 1 19:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:50:45.224439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f61b00-fadd-458d-8235-5b9263cb37f2 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:45.224837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f61b00-fadd-458d-8235-5b9263cb37f2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:45.224992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f61b00-fadd-458d-8235-5b9263cb37f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:45.242859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-43f61b00-fadd-458d-8235-5b9263cb37f2 None None] MFA Rules not processed for user `bba7d9205e694f2182d6091061d1219c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:50:45.244036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-43f61b00-fadd-458d-8235-5b9263cb37f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:50:45.252371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 758/1520] 10.222.0.66 () {58 vars in 1057 bytes} [Fri Jul 1 19:50:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:50:45.259666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28cee3-4a5e-4861-afca-a023de559d6c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:50:45.273442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28cee3-4a5e-4861-afca-a023de559d6c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:50:45.279013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05ad68-ce81-488f-ae3f-9f5df862d73c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:50:45.279898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05ad68-ce81-488f-ae3f-9f5df862d73c service placement] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:45.280174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05ad68-ce81-488f-ae3f-9f5df862d73c service placement] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:45.280444 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05ad68-ce81-488f-ae3f-9f5df862d73c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:45.288573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05ad68-ce81-488f-ae3f-9f5df862d73c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:50:45.290187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05ad68-ce81-488f-ae3f-9f5df862d73c service placement] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:50:45.299646 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 763/1521] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:50:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:50:51.660179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a483f66-3c9a-4844-b998-e1a245923079 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:51.660706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a483f66-3c9a-4844-b998-e1a245923079 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:51.660706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a483f66-3c9a-4844-b998-e1a245923079 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:51.662162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:50:51.666872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88feb8-89b7-47dd-bdfc-0828ff6b6353 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:51.667274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88feb8-89b7-47dd-bdfc-0828ff6b6353 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:51.667428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88feb8-89b7-47dd-bdfc-0828ff6b6353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:51.685839 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ab88feb8-89b7-47dd-bdfc-0828ff6b6353 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:50:51.687038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ab88feb8-89b7-47dd-bdfc-0828ff6b6353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:50:51.694798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:50:51.851011 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a483f66-3c9a-4844-b998-e1a245923079 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:50:51.865630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a483f66-3c9a-4844-b998-e1a245923079 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:50:51.871042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e90dc-6ec0-47ac-8fea-8acaa04e507a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:50:51.872178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e90dc-6ec0-47ac-8fea-8acaa04e507a service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:51.872535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e90dc-6ec0-47ac-8fea-8acaa04e507a service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:51.872814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e90dc-6ec0-47ac-8fea-8acaa04e507a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:51.881005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e90dc-6ec0-47ac-8fea-8acaa04e507a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:50:51.882625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e90dc-6ec0-47ac-8fea-8acaa04e507a service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:50:51.892711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 760/1524] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:50:52.711141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275f8d3-1689-4b06-8101-c42a93bddf38 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:52.711701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275f8d3-1689-4b06-8101-c42a93bddf38 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:52.711701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275f8d3-1689-4b06-8101-c42a93bddf38 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:52.713074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 19:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:50:52.717914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0663c5-4674-4564-b48e-d1086b7a44c5 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:52.718319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0663c5-4674-4564-b48e-d1086b7a44c5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:52.718469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0663c5-4674-4564-b48e-d1086b7a44c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:52.736591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-3e0663c5-4674-4564-b48e-d1086b7a44c5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:50:52.737815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3e0663c5-4674-4564-b48e-d1086b7a44c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:50:52.745807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 19:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:50:52.900238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275f8d3-1689-4b06-8101-c42a93bddf38 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:50:52.914878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275f8d3-1689-4b06-8101-c42a93bddf38 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:50:52.921263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d4cd6-6df4-4297-bc68-b3545a314d79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:50:52.921925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d4cd6-6df4-4297-bc68-b3545a314d79 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:52.922200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d4cd6-6df4-4297-bc68-b3545a314d79 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:52.922476 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d4cd6-6df4-4297-bc68-b3545a314d79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:52.930475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d4cd6-6df4-4297-bc68-b3545a314d79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:50:52.932012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d4cd6-6df4-4297-bc68-b3545a314d79 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:50:52.942314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 766/1527] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:50:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:50:57.133018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b66aa-de75-46d4-9852-5d73d7095abf None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:57.133567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b66aa-de75-46d4-9852-5d73d7095abf None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:57.133567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b66aa-de75-46d4-9852-5d73d7095abf None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:57.134963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:50:57.141294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1f7c3-5bb6-45fb-a818-ad345ba21323 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:57.141742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1f7c3-5bb6-45fb-a818-ad345ba21323 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:57.141879 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1f7c3-5bb6-45fb-a818-ad345ba21323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:57.160555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-04b1f7c3-5bb6-45fb-a818-ad345ba21323 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:50:57.161800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-04b1f7c3-5bb6-45fb-a818-ad345ba21323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:50:57.170348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:50:57.206882 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b66aa-de75-46d4-9852-5d73d7095abf None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:50:57.208572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-628b66aa-de75-46d4-9852-5d73d7095abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:50:57.266035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b66aa-de75-46d4-9852-5d73d7095abf None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:50:57.269541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f70f5-ea39-406d-a530-a3a11463057d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:50:57.270206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f70f5-ea39-406d-a530-a3a11463057d service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:57.270489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f70f5-ea39-406d-a530-a3a11463057d service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:57.270761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f70f5-ea39-406d-a530-a3a11463057d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:57.279861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f70f5-ea39-406d-a530-a3a11463057d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:50:57.281325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f70f5-ea39-406d-a530-a3a11463057d service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:50:57.286367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:50:57.291785 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 763/1530] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:50:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:50:59.293871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797f5e5-825a-4f68-9864-785b22ae8ae8 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:59.294402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797f5e5-825a-4f68-9864-785b22ae8ae8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:59.294500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797f5e5-825a-4f68-9864-785b22ae8ae8 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:59.295865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:50:59.300871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd371c-6991-401d-9c05-2a2dfc05e9f7 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:59.301310 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd371c-6991-401d-9c05-2a2dfc05e9f7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:59.301417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd371c-6991-401d-9c05-2a2dfc05e9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:59.320089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-2cfd371c-6991-401d-9c05-2a2dfc05e9f7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:50:59.321310 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2cfd371c-6991-401d-9c05-2a2dfc05e9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:50:59.330735 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:50:59.366770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797f5e5-825a-4f68-9864-785b22ae8ae8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:50:59.378601 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797f5e5-825a-4f68-9864-785b22ae8ae8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:50:59.382008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ef744-f76d-4778-935b-d4ee294fcbb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:50:59.382633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ef744-f76d-4778-935b-d4ee294fcbb0 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:50:59.382915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ef744-f76d-4778-935b-d4ee294fcbb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:50:59.383184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ef744-f76d-4778-935b-d4ee294fcbb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:50:59.392981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ef744-f76d-4778-935b-d4ee294fcbb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:50:59.394442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ef744-f76d-4778-935b-d4ee294fcbb0 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:50:59.403817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 769/1533] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:50:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:51:03.131807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9abf2-482a-4f6f-ad4c-c1cd7f38c5df None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:51:03.132370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9abf2-482a-4f6f-ad4c-c1cd7f38c5df None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:51:03.132370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9abf2-482a-4f6f-ad4c-c1cd7f38c5df None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:51:03.133744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:51:03.139411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296aa924-9ccf-45ab-b553-7700ad046da9 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:51:03.139797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296aa924-9ccf-45ab-b553-7700ad046da9 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:51:03.140000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296aa924-9ccf-45ab-b553-7700ad046da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:51:03.158005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-296aa924-9ccf-45ab-b553-7700ad046da9 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:51:03.159209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-296aa924-9ccf-45ab-b553-7700ad046da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:51:03.167300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:51:03.477223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9abf2-482a-4f6f-ad4c-c1cd7f38c5df None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:51:03.478904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-24c9abf2-482a-4f6f-ad4c-c1cd7f38c5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:51:03.538223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9abf2-482a-4f6f-ad4c-c1cd7f38c5df None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:51:03.541806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac74c-072b-4c23-8029-32c63be92d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:51:03.542448 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac74c-072b-4c23-8029-32c63be92d9e service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:51:03.543375 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac74c-072b-4c23-8029-32c63be92d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:51:03.543660 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac74c-072b-4c23-8029-32c63be92d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:51:03.552906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac74c-072b-4c23-8029-32c63be92d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:51:03.554385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac74c-072b-4c23-8029-32c63be92d9e service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:51:03.563818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 766/1536] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:51:05.666891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2ccd-9f55-4717-8d9f-81849bddf1f2 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:51:05.667448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2ccd-9f55-4717-8d9f-81849bddf1f2 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:51:05.667448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae2ccd-9f55-4717-8d9f-81849bddf1f2 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:51:05.669367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 19:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:51:05.674478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c62f7-21b2-4582-b6eb-3d915e29ad35 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:51:05.674861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c62f7-21b2-4582-b6eb-3d915e29ad35 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:51:05.675021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c62f7-21b2-4582-b6eb-3d915e29ad35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:51:05.693409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7f8c62f7-21b2-4582-b6eb-3d915e29ad35 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:51:05.694641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7f8c62f7-21b2-4582-b6eb-3d915e29ad35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:51:05.703238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 19:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:51:06.043208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae2ccd-9f55-4717-8d9f-81849bddf1f2 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:51:06.045398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1fae2ccd-9f55-4717-8d9f-81849bddf1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:51:06.058505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae2ccd-9f55-4717-8d9f-81849bddf1f2 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:51:06.062077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40824f05-84ae-48c6-8c73-bc859868fd6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:51:06.062710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824f05-84ae-48c6-8c73-bc859868fd6e service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:51:06.062992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824f05-84ae-48c6-8c73-bc859868fd6e service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:51:06.063268 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824f05-84ae-48c6-8c73-bc859868fd6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:51:06.072066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40824f05-84ae-48c6-8c73-bc859868fd6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:51:06.073481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40824f05-84ae-48c6-8c73-bc859868fd6e service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:51:06.082865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 772/1539] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:51:15.688288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=32 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:51:16.078717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=70 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:36.288687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b698-52d8-4a41-be81-5b1701c0a9e1 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.292180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b698-52d8-4a41-be81-5b1701c0a9e1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.292180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14b698-52d8-4a41-be81-5b1701c0a9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.314481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9a14b698-52d8-4a41-be81-5b1701c0a9e1 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:36.315698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9a14b698-52d8-4a41-be81-5b1701c0a9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:36.329446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 768/1540] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:52:36.338504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca2a09-e492-4560-939d-9e373cd7ac39 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.339623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca2a09-e492-4560-939d-9e373cd7ac39 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.339733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca2a09-e492-4560-939d-9e373cd7ac39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.368532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-dcca2a09-e492-4560-939d-9e373cd7ac39 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:36.369799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-dcca2a09-e492-4560-939d-9e373cd7ac39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:36.380911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 773/1541] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:36.386627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14b698-52d8-4a41-be81-5b1701c0a9e1 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:36.397191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14b698-52d8-4a41-be81-5b1701c0a9e1 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:36.399919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dedb16-1b30-4c10-8114-4f5acd2a8015 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.400211 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dedb16-1b30-4c10-8114-4f5acd2a8015 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.400491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dedb16-1b30-4c10-8114-4f5acd2a8015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.412524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e96392-63d2-42d1-a199-789829995712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:36.413231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e96392-63d2-42d1-a199-789829995712 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.413514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e96392-63d2-42d1-a199-789829995712 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.413818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e96392-63d2-42d1-a199-789829995712 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.416061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e96392-63d2-42d1-a199-789829995712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:36.416373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e96392-63d2-42d1-a199-789829995712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:36.418139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e96392-63d2-42d1-a199-789829995712 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:36.441568 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-99dedb16-1b30-4c10-8114-4f5acd2a8015 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:36.442842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-99dedb16-1b30-4c10-8114-4f5acd2a8015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:36.452356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 769/1542] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:36.466419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e96392-63d2-42d1-a199-789829995712 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:36.474109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 774/1543] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:52:36.478867 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e96392-63d2-42d1-a199-789829995712 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:36.481487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198952e0-2ada-4e0d-bd33-13c890963cbc None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.481811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198952e0-2ada-4e0d-bd33-13c890963cbc None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.482093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198952e0-2ada-4e0d-bd33-13c890963cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.498115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23b3f8-a913-4f49-965a-6e0f48350d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:36.498810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23b3f8-a913-4f49-965a-6e0f48350d5b admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.499090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23b3f8-a913-4f49-965a-6e0f48350d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.499365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23b3f8-a913-4f49-965a-6e0f48350d5b admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.500080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23b3f8-a913-4f49-965a-6e0f48350d5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:36.501136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: WARNING py.warnings [None req-cf23b3f8-a913-4f49-965a-6e0f48350d5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:52:36.501136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: warnings.warn(msg) Jul 01 19:52:36.501136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: Jul 01 19:52:36.502376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23b3f8-a913-4f49-965a-6e0f48350d5b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:36.532987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-198952e0-2ada-4e0d-bd33-13c890963cbc None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:36.534457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-198952e0-2ada-4e0d-bd33-13c890963cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:36.548925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 775/1544] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:52:36.555855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198952e0-2ada-4e0d-bd33-13c890963cbc None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:36.577539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198952e0-2ada-4e0d-bd33-13c890963cbc None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:36.587321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 770/1545] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/projects => generated 389 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:52:36.593255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23b3f8-a913-4f49-965a-6e0f48350d5b admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:36.727756 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0812d01-e9dd-476d-a4f8-7bc80603f2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:36.729010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0812d01-e9dd-476d-a4f8-7bc80603f2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.729614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0812d01-e9dd-476d-a4f8-7bc80603f2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.730735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0812d01-e9dd-476d-a4f8-7bc80603f2c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.732234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0812d01-e9dd-476d-a4f8-7bc80603f2c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:36.732771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0812d01-e9dd-476d-a4f8-7bc80603f2c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:36.736911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: WARNING py.warnings [None req-f0812d01-e9dd-476d-a4f8-7bc80603f2c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:52:36.736911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: warnings.warn(msg) Jul 01 19:52:36.736911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: Jul 01 19:52:36.742810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0812d01-e9dd-476d-a4f8-7bc80603f2c1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:36.759239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23b3f8-a913-4f49-965a-6e0f48350d5b admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:36.764874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 776/1546] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:36.766306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a492c3-49f7-406f-99c9-a32c1dc490ca None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.766591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a492c3-49f7-406f-99c9-a32c1dc490ca None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.766869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a492c3-49f7-406f-99c9-a32c1dc490ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.793900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e5a492c3-49f7-406f-99c9-a32c1dc490ca None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:36.794285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851767d1-c24c-4d11-9230-eb11e08eb8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:36.794992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851767d1-c24c-4d11-9230-eb11e08eb8fa admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.795274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851767d1-c24c-4d11-9230-eb11e08eb8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.795475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e5a492c3-49f7-406f-99c9-a32c1dc490ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:36.795598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851767d1-c24c-4d11-9230-eb11e08eb8fa admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.796243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851767d1-c24c-4d11-9230-eb11e08eb8fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:36.798491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851767d1-c24c-4d11-9230-eb11e08eb8fa admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:36.813371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 777/1547] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:36.814705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87553f01-7eb1-471e-bdf3-c0e7d55d0b7b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.814994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87553f01-7eb1-471e-bdf3-c0e7d55d0b7b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.815273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87553f01-7eb1-471e-bdf3-c0e7d55d0b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.845063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-87553f01-7eb1-471e-bdf3-c0e7d55d0b7b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:36.846193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 771/1548] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/users => generated 295 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:52:36.846328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-87553f01-7eb1-471e-bdf3-c0e7d55d0b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:36.847022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851767d1-c24c-4d11-9230-eb11e08eb8fa admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:36.858452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851767d1-c24c-4d11-9230-eb11e08eb8fa admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:36.859427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 778/1549] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:36.860697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a237d90-a71b-437c-a683-069203218e82 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.860981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a237d90-a71b-437c-a683-069203218e82 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.861253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a237d90-a71b-437c-a683-069203218e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.863765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b419be5-066f-4c6d-86b9-9990c199e2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:36.864614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b419be5-066f-4c6d-86b9-9990c199e2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.864897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b419be5-066f-4c6d-86b9-9990c199e2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.865180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b419be5-066f-4c6d-86b9-9990c199e2f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.865924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b419be5-066f-4c6d-86b9-9990c199e2f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:36.867838 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b419be5-066f-4c6d-86b9-9990c199e2f8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:36.890006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7a237d90-a71b-437c-a683-069203218e82 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:36.891329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7a237d90-a71b-437c-a683-069203218e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:36.903994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 779/1550] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:36.909867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65f36-37cb-4fa6-99c4-0a4cc2f6d011 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.909867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65f36-37cb-4fa6-99c4-0a4cc2f6d011 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.909867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65f36-37cb-4fa6-99c4-0a4cc2f6d011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.915793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 772/1551] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/projects => generated 363 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:36.917968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b2439-96b8-4a79-abb8-607a4152e04d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:36.917968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b2439-96b8-4a79-abb8-607a4152e04d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:36.918285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b2439-96b8-4a79-abb8-607a4152e04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:36.934149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-bbe65f36-37cb-4fa6-99c4-0a4cc2f6d011 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:36.935395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-bbe65f36-37cb-4fa6-99c4-0a4cc2f6d011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:36.937536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-b72b2439-96b8-4a79-abb8-607a4152e04d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:36.938841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b72b2439-96b8-4a79-abb8-607a4152e04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:37.003562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 773/1552] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:37.004824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b2439-96b8-4a79-abb8-607a4152e04d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.016454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b2439-96b8-4a79-abb8-607a4152e04d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.021162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed4c59-c7b4-48a5-945f-1b0f862adca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.021886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4c59-c7b4-48a5-945f-1b0f862adca4 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.022164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4c59-c7b4-48a5-945f-1b0f862adca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.022596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4c59-c7b4-48a5-945f-1b0f862adca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.023342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed4c59-c7b4-48a5-945f-1b0f862adca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.025016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed4c59-c7b4-48a5-945f-1b0f862adca4 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.037453 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 774/1553] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:37.038845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e3c60-bd63-41c4-a56f-4b51a526c5f9 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.039133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e3c60-bd63-41c4-a56f-4b51a526c5f9 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.039404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e3c60-bd63-41c4-a56f-4b51a526c5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.059616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d46e3c60-bd63-41c4-a56f-4b51a526c5f9 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:37.060844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d46e3c60-bd63-41c4-a56f-4b51a526c5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:37.068994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 775/1554] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:37.070345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d3f2-83fe-45be-b93e-db7b707dd273 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.070634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d3f2-83fe-45be-b93e-db7b707dd273 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.070902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d3f2-83fe-45be-b93e-db7b707dd273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.091518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-4e22d3f2-83fe-45be-b93e-db7b707dd273 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:37.092729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4e22d3f2-83fe-45be-b93e-db7b707dd273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:37.101011 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 776/1555] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:37.102206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c71592-8a82-4b43-8f05-f420717bc639 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.102486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c71592-8a82-4b43-8f05-f420717bc639 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.102757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c71592-8a82-4b43-8f05-f420717bc639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.109851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:37.123853 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 780/1556] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:37.123853 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe65f36-37cb-4fa6-99c4-0a4cc2f6d011 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.124143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d7c71592-8a82-4b43-8f05-f420717bc639 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:37.124143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d7c71592-8a82-4b43-8f05-f420717bc639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:37.130564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe65f36-37cb-4fa6-99c4-0a4cc2f6d011 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.131415 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 777/1557] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:37.132619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46b6be-07e6-4a10-9329-317e60daa183 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.132899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46b6be-07e6-4a10-9329-317e60daa183 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.133170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46b6be-07e6-4a10-9329-317e60daa183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.139121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b24fea-5da0-4713-97f7-71a1c377fbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.139779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b24fea-5da0-4713-97f7-71a1c377fbac admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.140066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b24fea-5da0-4713-97f7-71a1c377fbac admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.140340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b24fea-5da0-4713-97f7-71a1c377fbac admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.141024 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b24fea-5da0-4713-97f7-71a1c377fbac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.142066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: WARNING py.warnings [None req-83b24fea-5da0-4713-97f7-71a1c377fbac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:52:37.142066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: warnings.warn(msg) Jul 01 19:52:37.142066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: Jul 01 19:52:37.143261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b24fea-5da0-4713-97f7-71a1c377fbac admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.151280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-0d46b6be-07e6-4a10-9329-317e60daa183 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:37.154826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0d46b6be-07e6-4a10-9329-317e60daa183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:37.162113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 778/1558] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:37.163316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6f2d6-6c0c-4eb1-afed-4d1f1bf84860 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.163592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6f2d6-6c0c-4eb1-afed-4d1f1bf84860 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.163869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6f2d6-6c0c-4eb1-afed-4d1f1bf84860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.166425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 781/1559] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/users => generated 282 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:37.167232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b24fea-5da0-4713-97f7-71a1c377fbac admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.176765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b24fea-5da0-4713-97f7-71a1c377fbac admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.181330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceec0cb-a740-431e-8c8b-0594f131d59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.182009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceec0cb-a740-431e-8c8b-0594f131d59e admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.182289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceec0cb-a740-431e-8c8b-0594f131d59e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.183129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceec0cb-a740-431e-8c8b-0594f131d59e admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.184012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceec0cb-a740-431e-8c8b-0594f131d59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.185699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceec0cb-a740-431e-8c8b-0594f131d59e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.186037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-abb6f2d6-6c0c-4eb1-afed-4d1f1bf84860 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:37.187211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-abb6f2d6-6c0c-4eb1-afed-4d1f1bf84860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:37.195229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 779/1560] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:37.196296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6f2d6-6c0c-4eb1-afed-4d1f1bf84860 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.201472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 782/1561] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:37.203361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2baecb-0160-47f7-bafb-89b03cba605d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.203781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2baecb-0160-47f7-bafb-89b03cba605d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.204192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2baecb-0160-47f7-bafb-89b03cba605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.207531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6f2d6-6c0c-4eb1-afed-4d1f1bf84860 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.212363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68877ca6-5335-499a-9c25-d4730c11a76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.213032 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68877ca6-5335-499a-9c25-d4730c11a76d admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.213320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68877ca6-5335-499a-9c25-d4730c11a76d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.213625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68877ca6-5335-499a-9c25-d4730c11a76d admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.214419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68877ca6-5335-499a-9c25-d4730c11a76d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:37.214735 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68877ca6-5335-499a-9c25-d4730c11a76d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.216351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68877ca6-5335-499a-9c25-d4730c11a76d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.228299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-3f2baecb-0160-47f7-bafb-89b03cba605d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:37.230343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3f2baecb-0160-47f7-bafb-89b03cba605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:37.234888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 780/1562] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:37.235809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68877ca6-5335-499a-9c25-d4730c11a76d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.242323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 783/1563] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:37.242323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2baecb-0160-47f7-bafb-89b03cba605d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.248736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68877ca6-5335-499a-9c25-d4730c11a76d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.251997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2baecb-0160-47f7-bafb-89b03cba605d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.253504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d53216-7f84-4be8-a392-a8ca75c3514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.254186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d53216-7f84-4be8-a392-a8ca75c3514c admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.254468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d53216-7f84-4be8-a392-a8ca75c3514c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.254743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d53216-7f84-4be8-a392-a8ca75c3514c admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.255466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d53216-7f84-4be8-a392-a8ca75c3514c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:37.255770 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d53216-7f84-4be8-a392-a8ca75c3514c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.256489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb51e3-d8f0-4f67-8613-e08987b5c383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.257125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb51e3-d8f0-4f67-8613-e08987b5c383 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.257377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d53216-7f84-4be8-a392-a8ca75c3514c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.257516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb51e3-d8f0-4f67-8613-e08987b5c383 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.257699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb51e3-d8f0-4f67-8613-e08987b5c383 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.258412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb51e3-d8f0-4f67-8613-e08987b5c383 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:37.258865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb51e3-d8f0-4f67-8613-e08987b5c383 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.260290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb51e3-d8f0-4f67-8613-e08987b5c383 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.270023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 781/1564] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:37.271334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8fd89-4d40-4c60-adf1-4b6e86e528f4 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.271626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8fd89-4d40-4c60-adf1-4b6e86e528f4 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.271910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8fd89-4d40-4c60-adf1-4b6e86e528f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.278533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 784/1565] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:37.278533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb51e3-d8f0-4f67-8613-e08987b5c383 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.288198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb51e3-d8f0-4f67-8613-e08987b5c383 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.293427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7fbfd-1e76-46e6-bba0-5e464e1e9171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.294132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7fbfd-1e76-46e6-bba0-5e464e1e9171 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.294412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7fbfd-1e76-46e6-bba0-5e464e1e9171 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.294686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7fbfd-1e76-46e6-bba0-5e464e1e9171 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.295431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7fbfd-1e76-46e6-bba0-5e464e1e9171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.295818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-90c8fd89-4d40-4c60-adf1-4b6e86e528f4 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:37.297008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-90c8fd89-4d40-4c60-adf1-4b6e86e528f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:37.297167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7fbfd-1e76-46e6-bba0-5e464e1e9171 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.298590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:37.305348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 782/1566] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:37.306252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8fd89-4d40-4c60-adf1-4b6e86e528f4 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.314075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 785/1567] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:37.315318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7fbfd-1e76-46e6-bba0-5e464e1e9171 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.315583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8fd89-4d40-4c60-adf1-4b6e86e528f4 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.320080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1e173-1ec6-4f9d-bcca-54ad9d1f080d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.320722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e173-1ec6-4f9d-bcca-54ad9d1f080d admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.320998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e173-1ec6-4f9d-bcca-54ad9d1f080d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.321274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e173-1ec6-4f9d-bcca-54ad9d1f080d admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.322023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1e173-1ec6-4f9d-bcca-54ad9d1f080d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:37.322324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1e173-1ec6-4f9d-bcca-54ad9d1f080d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.323912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1e173-1ec6-4f9d-bcca-54ad9d1f080d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.325510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7fbfd-1e76-46e6-bba0-5e464e1e9171 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.330061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68504365-c6f5-49f8-91f9-5db0b4859df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.330914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68504365-c6f5-49f8-91f9-5db0b4859df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.331186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68504365-c6f5-49f8-91f9-5db0b4859df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.331450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68504365-c6f5-49f8-91f9-5db0b4859df3 admin admin] PATH_INFO: `/v3/projects/b8e38ec3f92a47fe98829eb3d289cdb4/users/3307be3159654f55be46e75c82584bf0/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.335796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 783/1568] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:37.336576 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1e173-1ec6-4f9d-bcca-54ad9d1f080d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.346665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1e173-1ec6-4f9d-bcca-54ad9d1f080d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.349762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68504365-c6f5-49f8-91f9-5db0b4859df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e38ec3f92a47fe98829eb3d289cdb4, user_id=3307be3159654f55be46e75c82584bf0, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.351499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c22a5d-0ed4-4f92-8d1b-e1457034573c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.352082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c22a5d-0ed4-4f92-8d1b-e1457034573c admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.352361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c22a5d-0ed4-4f92-8d1b-e1457034573c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.352476 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68504365-c6f5-49f8-91f9-5db0b4859df3 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.352631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c22a5d-0ed4-4f92-8d1b-e1457034573c admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.353335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c22a5d-0ed4-4f92-8d1b-e1457034573c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.355961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c22a5d-0ed4-4f92-8d1b-e1457034573c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.366865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 786/1569] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:37 2022] PUT /identity/v3/projects/b8e38ec3f92a47fe98829eb3d289cdb4/users/3307be3159654f55be46e75c82584bf0/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:37.367679 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68504365-c6f5-49f8-91f9-5db0b4859df3 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.386744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 784/1570] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:37.387561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c22a5d-0ed4-4f92-8d1b-e1457034573c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.434347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68504365-c6f5-49f8-91f9-5db0b4859df3 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.450850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c22a5d-0ed4-4f92-8d1b-e1457034573c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.455536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35665c1a-6a57-4ad4-a16a-5e82e04c6436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.456187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35665c1a-6a57-4ad4-a16a-5e82e04c6436 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.456463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35665c1a-6a57-4ad4-a16a-5e82e04c6436 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.456733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35665c1a-6a57-4ad4-a16a-5e82e04c6436 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.457443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35665c1a-6a57-4ad4-a16a-5e82e04c6436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.459368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35665c1a-6a57-4ad4-a16a-5e82e04c6436 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.485985 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 785/1571] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:37.486834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35665c1a-6a57-4ad4-a16a-5e82e04c6436 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.491176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d021c-f251-4638-8993-32744c1834f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.491859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d021c-f251-4638-8993-32744c1834f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.492133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d021c-f251-4638-8993-32744c1834f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.492402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d021c-f251-4638-8993-32744c1834f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.493118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d021c-f251-4638-8993-32744c1834f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:37.493415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d021c-f251-4638-8993-32744c1834f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.496411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d021c-f251-4638-8993-32744c1834f2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.508136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 787/1572] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:37.508136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d021c-f251-4638-8993-32744c1834f2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.521694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d021c-f251-4638-8993-32744c1834f2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.549662 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35665c1a-6a57-4ad4-a16a-5e82e04c6436 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.555153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cbfc4-8e25-40a3-b86f-ef440f8a2c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.555970 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cbfc4-8e25-40a3-b86f-ef440f8a2c3b admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.556247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cbfc4-8e25-40a3-b86f-ef440f8a2c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.556519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cbfc4-8e25-40a3-b86f-ef440f8a2c3b admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.557202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cbfc4-8e25-40a3-b86f-ef440f8a2c3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.560836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cbfc4-8e25-40a3-b86f-ef440f8a2c3b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.564416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:37.587451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98137bb-4162-4812-8e6a-1d091ff49d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.588119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98137bb-4162-4812-8e6a-1d091ff49d22 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.588400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98137bb-4162-4812-8e6a-1d091ff49d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.588673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98137bb-4162-4812-8e6a-1d091ff49d22 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.589714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98137bb-4162-4812-8e6a-1d091ff49d22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:37.590023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98137bb-4162-4812-8e6a-1d091ff49d22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.591002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 786/1573] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:37.592133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98137bb-4162-4812-8e6a-1d091ff49d22 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.592433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cbfc4-8e25-40a3-b86f-ef440f8a2c3b admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.611573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 788/1574] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:37.611573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98137bb-4162-4812-8e6a-1d091ff49d22 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.615769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98137bb-4162-4812-8e6a-1d091ff49d22 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.619342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:37.656967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cbfc4-8e25-40a3-b86f-ef440f8a2c3b admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.662341 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279dcfd-96f6-4c44-ab3c-ff0f59d34b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.663002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279dcfd-96f6-4c44-ab3c-ff0f59d34b71 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.663288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279dcfd-96f6-4c44-ab3c-ff0f59d34b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.663560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279dcfd-96f6-4c44-ab3c-ff0f59d34b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.664377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279dcfd-96f6-4c44-ab3c-ff0f59d34b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.666018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279dcfd-96f6-4c44-ab3c-ff0f59d34b71 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.676101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239ebdb-c90c-4f39-8363-ae8d74c58f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.676687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 787/1575] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:37.676815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239ebdb-c90c-4f39-8363-ae8d74c58f85 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.677026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239ebdb-c90c-4f39-8363-ae8d74c58f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.677300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239ebdb-c90c-4f39-8363-ae8d74c58f85 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.677836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621257b-4cbd-4381-8dbc-55015a944c91 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.678019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239ebdb-c90c-4f39-8363-ae8d74c58f85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.678122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621257b-4cbd-4381-8dbc-55015a944c91 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.678386 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621257b-4cbd-4381-8dbc-55015a944c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.679997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239ebdb-c90c-4f39-8363-ae8d74c58f85 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.695557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c621257b-4cbd-4381-8dbc-55015a944c91 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:37.696813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c621257b-4cbd-4381-8dbc-55015a944c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:37.705289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 788/1576] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:37.708733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 789/1577] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:37.708733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239ebdb-c90c-4f39-8363-ae8d74c58f85 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.708930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621257b-4cbd-4381-8dbc-55015a944c91 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.720228 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621257b-4cbd-4381-8dbc-55015a944c91 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.775887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239ebdb-c90c-4f39-8363-ae8d74c58f85 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.776389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40588e-3a2e-4367-9b0b-2d827b2b20cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.777074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40588e-3a2e-4367-9b0b-2d827b2b20cf admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.777357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40588e-3a2e-4367-9b0b-2d827b2b20cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.777662 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40588e-3a2e-4367-9b0b-2d827b2b20cf admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.778631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40588e-3a2e-4367-9b0b-2d827b2b20cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.780650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40588e-3a2e-4367-9b0b-2d827b2b20cf admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.780925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3186794-1a67-4530-ab4d-5720db0115e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.781602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3186794-1a67-4530-ab4d-5720db0115e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.781948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3186794-1a67-4530-ab4d-5720db0115e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.782168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3186794-1a67-4530-ab4d-5720db0115e3 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.782849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3186794-1a67-4530-ab4d-5720db0115e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.784746 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3186794-1a67-4530-ab4d-5720db0115e3 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.806268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 789/1578] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:37.806268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40588e-3a2e-4367-9b0b-2d827b2b20cf admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.806569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 790/1579] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:37.806569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3186794-1a67-4530-ab4d-5720db0115e3 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.814968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40588e-3a2e-4367-9b0b-2d827b2b20cf admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.816721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3186794-1a67-4530-ab4d-5720db0115e3 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.819593 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ccdc-a844-417a-822c-755c76c5347b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.820229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ccdc-a844-417a-822c-755c76c5347b admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.820543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ccdc-a844-417a-822c-755c76c5347b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.820822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ccdc-a844-417a-822c-755c76c5347b admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.821499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90252a-9969-4528-afe2-ab9f06dab720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.821732 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731ccdc-a844-417a-822c-755c76c5347b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.822170 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90252a-9969-4528-afe2-ab9f06dab720 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.822449 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90252a-9969-4528-afe2-ab9f06dab720 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.822719 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90252a-9969-4528-afe2-ab9f06dab720 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.823433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731ccdc-a844-417a-822c-755c76c5347b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.823555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90252a-9969-4528-afe2-ab9f06dab720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.825517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90252a-9969-4528-afe2-ab9f06dab720 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.849236 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 791/1580] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/users => generated 284 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:37.849873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 790/1581] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:37.850645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ccdc-a844-417a-822c-755c76c5347b admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.851652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90252a-9969-4528-afe2-ab9f06dab720 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.868178 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90252a-9969-4528-afe2-ab9f06dab720 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.902878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:37.922784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ccdc-a844-417a-822c-755c76c5347b admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:37.927406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ab837-8ccf-4d67-8ead-a1c0330c08cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.928298 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ab837-8ccf-4d67-8ead-a1c0330c08cf admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.928589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ab837-8ccf-4d67-8ead-a1c0330c08cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.928942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0e37a-7b35-4db5-be12-431253417f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:37.929068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ab837-8ccf-4d67-8ead-a1c0330c08cf admin admin] PATH_INFO: `/v3/projects/3a0a5970f9994cc690afeb48cecb4b04/users/0b16b4ddea3a43489100e90281741160/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.929625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0e37a-7b35-4db5-be12-431253417f66 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:37.929960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0e37a-7b35-4db5-be12-431253417f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:37.930345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0e37a-7b35-4db5-be12-431253417f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:37.931050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0e37a-7b35-4db5-be12-431253417f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.933307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0e37a-7b35-4db5-be12-431253417f66 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.941444 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:37.944293 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ab837-8ccf-4d67-8ead-a1c0330c08cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a5970f9994cc690afeb48cecb4b04, user_id=0b16b4ddea3a43489100e90281741160, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:37.945343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: WARNING py.warnings [None req-7e7ab837-8ccf-4d67-8ead-a1c0330c08cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:52:37.945343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: warnings.warn(msg) Jul 01 19:52:37.945343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: Jul 01 19:52:37.947298 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ab837-8ccf-4d67-8ead-a1c0330c08cf admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:37.962697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 791/1582] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:37 2022] PUT /identity/v3/projects/3a0a5970f9994cc690afeb48cecb4b04/users/0b16b4ddea3a43489100e90281741160/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:37.963329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ab837-8ccf-4d67-8ead-a1c0330c08cf admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:37.963654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 792/1583] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:37.965046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0e37a-7b35-4db5-be12-431253417f66 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.024649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0e37a-7b35-4db5-be12-431253417f66 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.029110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ce8fa-e879-42b8-ae0f-a7659b1983be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.029790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ce8fa-e879-42b8-ae0f-a7659b1983be service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.030074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ce8fa-e879-42b8-ae0f-a7659b1983be service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.030357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ce8fa-e879-42b8-ae0f-a7659b1983be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.033973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ab837-8ccf-4d67-8ead-a1c0330c08cf admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.089936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f97154-c91a-408e-8450-73149cb33fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.090573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ce8fa-e879-42b8-ae0f-a7659b1983be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.090687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f97154-c91a-408e-8450-73149cb33fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.090878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f97154-c91a-408e-8450-73149cb33fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.091150 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f97154-c91a-408e-8450-73149cb33fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.091844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f97154-c91a-408e-8450-73149cb33fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.092005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ce8fa-e879-42b8-ae0f-a7659b1983be service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.093971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f97154-c91a-408e-8450-73149cb33fc1 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.101574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 793/1584] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:38.102406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ce8fa-e879-42b8-ae0f-a7659b1983be service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.113679 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ce8fa-e879-42b8-ae0f-a7659b1983be service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.115879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 792/1585] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:37 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:38.116760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f97154-c91a-408e-8450-73149cb33fc1 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.119404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77e310-c366-476f-aa97-19962869c63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.120356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e310-c366-476f-aa97-19962869c63d admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.120647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e310-c366-476f-aa97-19962869c63d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.120948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77e310-c366-476f-aa97-19962869c63d admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.121716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77e310-c366-476f-aa97-19962869c63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.123335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77e310-c366-476f-aa97-19962869c63d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.128745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f97154-c91a-408e-8450-73149cb33fc1 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.135112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade709-06f8-40c3-a279-9095f3910711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.138564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 794/1586] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.138790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade709-06f8-40c3-a279-9095f3910711 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.138790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade709-06f8-40c3-a279-9095f3910711 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.138790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ade709-06f8-40c3-a279-9095f3910711 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.139064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77e310-c366-476f-aa97-19962869c63d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.139166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ade709-06f8-40c3-a279-9095f3910711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.139166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ade709-06f8-40c3-a279-9095f3910711 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.149677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77e310-c366-476f-aa97-19962869c63d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.154259 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 793/1587] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.155152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade709-06f8-40c3-a279-9095f3910711 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.156919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2814-b380-451b-90e9-f0f5e901771e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.157674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2814-b380-451b-90e9-f0f5e901771e admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.157960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2814-b380-451b-90e9-f0f5e901771e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.158237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab2814-b380-451b-90e9-f0f5e901771e admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.159000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab2814-b380-451b-90e9-f0f5e901771e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.160638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab2814-b380-451b-90e9-f0f5e901771e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.165515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ade709-06f8-40c3-a279-9095f3910711 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.174640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 795/1588] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.174825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e429f-0b71-4ace-b882-138ae6edefdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.174937 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2814-b380-451b-90e9-f0f5e901771e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.175030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e429f-0b71-4ace-b882-138ae6edefdf admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.175030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e429f-0b71-4ace-b882-138ae6edefdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.175030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e429f-0b71-4ace-b882-138ae6edefdf admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.175030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e429f-0b71-4ace-b882-138ae6edefdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.176498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e429f-0b71-4ace-b882-138ae6edefdf admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.187512 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab2814-b380-451b-90e9-f0f5e901771e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.194955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b7e29-fc48-4dd4-958d-de7793a4c057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.196722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b7e29-fc48-4dd4-958d-de7793a4c057 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.196722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b7e29-fc48-4dd4-958d-de7793a4c057 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.196722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b7e29-fc48-4dd4-958d-de7793a4c057 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.197927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b7e29-fc48-4dd4-958d-de7793a4c057 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.201126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b7e29-fc48-4dd4-958d-de7793a4c057 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.201289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 794/1589] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:38 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:38.202818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a91b8-54e2-4ff2-b584-3a148906476b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.203173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a91b8-54e2-4ff2-b584-3a148906476b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.203386 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a91b8-54e2-4ff2-b584-3a148906476b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.226971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 796/1590] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:38 2022] POST /identity/v3/users => generated 288 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:38.226971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b7e29-fc48-4dd4-958d-de7793a4c057 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.234029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-0f5a91b8-54e2-4ff2-b584-3a148906476b None None] MFA Rules not processed for user `0b16b4ddea3a43489100e90281741160`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:38.235355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0f5a91b8-54e2-4ff2-b584-3a148906476b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:38.239312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b7e29-fc48-4dd4-958d-de7793a4c057 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.245500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38c348-5afd-40d4-b063-638b71bc6c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.246657 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38c348-5afd-40d4-b063-638b71bc6c38 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.247139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38c348-5afd-40d4-b063-638b71bc6c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.247621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38c348-5afd-40d4-b063-638b71bc6c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.248432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38c348-5afd-40d4-b063-638b71bc6c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.250384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38c348-5afd-40d4-b063-638b71bc6c38 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.257887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:38.262651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 797/1591] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.263367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38c348-5afd-40d4-b063-638b71bc6c38 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.274762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38c348-5afd-40d4-b063-638b71bc6c38 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.281515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250023f-f3cb-47ae-b7b0-404c1727a77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.282283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250023f-f3cb-47ae-b7b0-404c1727a77f admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.282634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250023f-f3cb-47ae-b7b0-404c1727a77f admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.283040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250023f-f3cb-47ae-b7b0-404c1727a77f admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.284052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250023f-f3cb-47ae-b7b0-404c1727a77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.285754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250023f-f3cb-47ae-b7b0-404c1727a77f admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.299318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 798/1592] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.300181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250023f-f3cb-47ae-b7b0-404c1727a77f admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.314813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250023f-f3cb-47ae-b7b0-404c1727a77f admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.320934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b4283-64c3-489b-add2-50597b57774d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.322019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b4283-64c3-489b-add2-50597b57774d admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.322299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b4283-64c3-489b-add2-50597b57774d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.322580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b4283-64c3-489b-add2-50597b57774d admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.324114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b4283-64c3-489b-add2-50597b57774d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.325893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b4283-64c3-489b-add2-50597b57774d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.326339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 795/1593] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:38 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:38.327202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a91b8-54e2-4ff2-b584-3a148906476b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.340748 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a91b8-54e2-4ff2-b584-3a148906476b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.341865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 799/1594] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.346421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b4283-64c3-489b-add2-50597b57774d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.347249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b75b68-f731-4f03-a031-5dd3bf7f261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.348006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b75b68-f731-4f03-a031-5dd3bf7f261f admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.348588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b75b68-f731-4f03-a031-5dd3bf7f261f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.348929 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b75b68-f731-4f03-a031-5dd3bf7f261f admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.349764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b75b68-f731-4f03-a031-5dd3bf7f261f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.352453 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b75b68-f731-4f03-a031-5dd3bf7f261f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.356842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b4283-64c3-489b-add2-50597b57774d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.361614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877f4a3-3bdf-4f85-b5c4-04112105f56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.362277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877f4a3-3bdf-4f85-b5c4-04112105f56c admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.362560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877f4a3-3bdf-4f85-b5c4-04112105f56c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.362836 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877f4a3-3bdf-4f85-b5c4-04112105f56c admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.363557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877f4a3-3bdf-4f85-b5c4-04112105f56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.365155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877f4a3-3bdf-4f85-b5c4-04112105f56c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.368603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 796/1595] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.368603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b75b68-f731-4f03-a031-5dd3bf7f261f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.376812 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 800/1596] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.377906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877f4a3-3bdf-4f85-b5c4-04112105f56c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.389734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877f4a3-3bdf-4f85-b5c4-04112105f56c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.390826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b75b68-f731-4f03-a031-5dd3bf7f261f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.394865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829a726-c86c-48e5-94d4-dca784eaa23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.395342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3a717-4c99-4673-aabf-f4b233e9e897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.395654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829a726-c86c-48e5-94d4-dca784eaa23a admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.395940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829a726-c86c-48e5-94d4-dca784eaa23a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.396049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3a717-4c99-4673-aabf-f4b233e9e897 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.396217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829a726-c86c-48e5-94d4-dca784eaa23a admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.396324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3a717-4c99-4673-aabf-f4b233e9e897 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.396529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3a717-4c99-4673-aabf-f4b233e9e897 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.397081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829a726-c86c-48e5-94d4-dca784eaa23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.397327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3a717-4c99-4673-aabf-f4b233e9e897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.398716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829a726-c86c-48e5-94d4-dca784eaa23a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.398946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3a717-4c99-4673-aabf-f4b233e9e897 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.410855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 797/1597] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.414421 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3a717-4c99-4673-aabf-f4b233e9e897 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.418708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 801/1598] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.418708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829a726-c86c-48e5-94d4-dca784eaa23a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.435292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3a717-4c99-4673-aabf-f4b233e9e897 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.436806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:38.443504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8937e1-01fd-4e6d-ac45-fac300ca0678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.444651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829a726-c86c-48e5-94d4-dca784eaa23a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.444887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8937e1-01fd-4e6d-ac45-fac300ca0678 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.445225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8937e1-01fd-4e6d-ac45-fac300ca0678 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.445709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8937e1-01fd-4e6d-ac45-fac300ca0678 admin admin] PATH_INFO: `/v3/projects/2347fd8e4a1740fdb98218aa30c150e0/users/71647f9f93ec4235818d27640058d25c/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.455446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69717ed0-570b-46dd-8cff-e45619950a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.459298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69717ed0-570b-46dd-8cff-e45619950a7d admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.459298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69717ed0-570b-46dd-8cff-e45619950a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.459298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69717ed0-570b-46dd-8cff-e45619950a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.459298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69717ed0-570b-46dd-8cff-e45619950a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.462687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69717ed0-570b-46dd-8cff-e45619950a7d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.472769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8937e1-01fd-4e6d-ac45-fac300ca0678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2347fd8e4a1740fdb98218aa30c150e0, user_id=71647f9f93ec4235818d27640058d25c, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.478029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8937e1-01fd-4e6d-ac45-fac300ca0678 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.506218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 798/1599] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:38 2022] PUT /identity/v3/projects/2347fd8e4a1740fdb98218aa30c150e0/users/71647f9f93ec4235818d27640058d25c/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:38.507637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8937e1-01fd-4e6d-ac45-fac300ca0678 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.508486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 802/1600] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:38 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:38.509834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69717ed0-570b-46dd-8cff-e45619950a7d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.600837 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69717ed0-570b-46dd-8cff-e45619950a7d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.601339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8937e1-01fd-4e6d-ac45-fac300ca0678 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.606944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08047933-5d3b-4ac0-8c8a-5069cd5f6779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.607110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ac092-50ac-482c-8569-e7f0e7fcc58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.608096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ac092-50ac-482c-8569-e7f0e7fcc58f admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.608336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08047933-5d3b-4ac0-8c8a-5069cd5f6779 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.608566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ac092-50ac-482c-8569-e7f0e7fcc58f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.608902 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08047933-5d3b-4ac0-8c8a-5069cd5f6779 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.609034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ac092-50ac-482c-8569-e7f0e7fcc58f admin admin] PATH_INFO: `/v3/projects/fcb6931bdfc84ca090b172a0fcf4f6ae/users/3f1f6d0d8a6d4c649b0f4f58de0d59d1/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.609137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08047933-5d3b-4ac0-8c8a-5069cd5f6779 admin admin] PATH_INFO: `/v3/projects/f15a8f44ad544e80982c570d6d12473b/users/67319ef6317d4c8cac2cde40d9fa9625/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.626020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08047933-5d3b-4ac0-8c8a-5069cd5f6779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15a8f44ad544e80982c570d6d12473b, user_id=67319ef6317d4c8cac2cde40d9fa9625, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.628886 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08047933-5d3b-4ac0-8c8a-5069cd5f6779 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.630007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ac092-50ac-482c-8569-e7f0e7fcc58f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb6931bdfc84ca090b172a0fcf4f6ae, user_id=3f1f6d0d8a6d4c649b0f4f58de0d59d1, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.633046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ac092-50ac-482c-8569-e7f0e7fcc58f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.645903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 803/1601] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:38 2022] PUT /identity/v3/projects/f15a8f44ad544e80982c570d6d12473b/users/67319ef6317d4c8cac2cde40d9fa9625/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:38.658223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08047933-5d3b-4ac0-8c8a-5069cd5f6779 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.661774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 799/1602] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:38 2022] PUT /identity/v3/projects/fcb6931bdfc84ca090b172a0fcf4f6ae/users/3f1f6d0d8a6d4c649b0f4f58de0d59d1/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:38.662686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ac092-50ac-482c-8569-e7f0e7fcc58f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.731378 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ac092-50ac-482c-8569-e7f0e7fcc58f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.733159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08047933-5d3b-4ac0-8c8a-5069cd5f6779 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.737031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb1d1e-8e80-447e-866d-110bb712fdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.737907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb1d1e-8e80-447e-866d-110bb712fdcd admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.738310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d6892-60b5-4ca9-8e20-d679e8352a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.738446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb1d1e-8e80-447e-866d-110bb712fdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.738792 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb1d1e-8e80-447e-866d-110bb712fdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.739160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d6892-60b5-4ca9-8e20-d679e8352a86 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.739508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d6892-60b5-4ca9-8e20-d679e8352a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.739710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb1d1e-8e80-447e-866d-110bb712fdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.739915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d6892-60b5-4ca9-8e20-d679e8352a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.740702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d6892-60b5-4ca9-8e20-d679e8352a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.741539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb1d1e-8e80-447e-866d-110bb712fdcd admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.742467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d6892-60b5-4ca9-8e20-d679e8352a86 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.743432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:38.756671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 804/1603] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.757476 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d6892-60b5-4ca9-8e20-d679e8352a86 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.757969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 800/1604] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:38.759469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256ec0e-521c-4283-9feb-6d8dd0d52298 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.760000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256ec0e-521c-4283-9feb-6d8dd0d52298 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.760369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256ec0e-521c-4283-9feb-6d8dd0d52298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.770632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d6892-60b5-4ca9-8e20-d679e8352a86 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.777856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca630a-25ba-45b3-813b-972a97a41a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.779240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca630a-25ba-45b3-813b-972a97a41a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.779586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9256ec0e-521c-4283-9feb-6d8dd0d52298 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:38.779705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca630a-25ba-45b3-813b-972a97a41a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.779971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca630a-25ba-45b3-813b-972a97a41a00 admin admin] PATH_INFO: `/v3/projects/025035b3b0b348d7acddfe456dd6a109/users/3198a63f6d1a4cdd88d76189658e79c0/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.783080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9256ec0e-521c-4283-9feb-6d8dd0d52298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:38.791817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 801/1605] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:38.792829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256ec0e-521c-4283-9feb-6d8dd0d52298 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.805622 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256ec0e-521c-4283-9feb-6d8dd0d52298 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:38.806471 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca630a-25ba-45b3-813b-972a97a41a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=025035b3b0b348d7acddfe456dd6a109, user_id=3198a63f6d1a4cdd88d76189658e79c0, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.809767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca630a-25ba-45b3-813b-972a97a41a00 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.811759 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfa6c5-a705-4714-b787-e6d1b8e83174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:38.812191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfa6c5-a705-4714-b787-e6d1b8e83174 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.812473 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfa6c5-a705-4714-b787-e6d1b8e83174 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.812750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfa6c5-a705-4714-b787-e6d1b8e83174 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.813447 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfa6c5-a705-4714-b787-e6d1b8e83174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:38.815915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfa6c5-a705-4714-b787-e6d1b8e83174 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:38.826358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 805/1606] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:38 2022] PUT /identity/v3/projects/025035b3b0b348d7acddfe456dd6a109/users/3198a63f6d1a4cdd88d76189658e79c0/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:38.831405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641ceb7-c3a0-421e-a2ae-f74921a6b546 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.831734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641ceb7-c3a0-421e-a2ae-f74921a6b546 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.832038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641ceb7-c3a0-421e-a2ae-f74921a6b546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.846308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 802/1607] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:38 2022] POST /identity/v3/users => generated 289 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:38.847715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451caafd-5f4e-43b8-8c8e-b0e6240f9ddd None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:38.848051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451caafd-5f4e-43b8-8c8e-b0e6240f9ddd None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:38.848428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451caafd-5f4e-43b8-8c8e-b0e6240f9ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:38.863327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-6641ceb7-c3a0-421e-a2ae-f74921a6b546 None None] MFA Rules not processed for user `67319ef6317d4c8cac2cde40d9fa9625`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:38.864867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6641ceb7-c3a0-421e-a2ae-f74921a6b546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:38.882141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-451caafd-5f4e-43b8-8c8e-b0e6240f9ddd None None] MFA Rules not processed for user `3f1f6d0d8a6d4c649b0f4f58de0d59d1`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:38.884477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-451caafd-5f4e-43b8-8c8e-b0e6240f9ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:38.900557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:38.965381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 806/1608] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:38.966149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641ceb7-c3a0-421e-a2ae-f74921a6b546 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:38.977721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 803/1609] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:38.978588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451caafd-5f4e-43b8-8c8e-b0e6240f9ddd None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.056487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451caafd-5f4e-43b8-8c8e-b0e6240f9ddd None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.062989 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3c079-f1e9-4c28-919f-c0a42e457c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.063899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3c079-f1e9-4c28-919f-c0a42e457c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.064299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3c079-f1e9-4c28-919f-c0a42e457c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.064463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3c079-f1e9-4c28-919f-c0a42e457c87 admin admin] PATH_INFO: `/v3/projects/8ff2d50730e24f3b9983e44416421b05/users/3e80908e6ccb48ec984fa11268437eaa/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.067523 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641ceb7-c3a0-421e-a2ae-f74921a6b546 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.073411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75c7d8-73c9-45d0-90be-b9de105e205e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.074540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75c7d8-73c9-45d0-90be-b9de105e205e admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.075571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75c7d8-73c9-45d0-90be-b9de105e205e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.075941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75c7d8-73c9-45d0-90be-b9de105e205e admin admin] PATH_INFO: `/v3/projects/0ba0ddfd3efd41ee9687efe014c40bf0/users/0267350339c44000bcc49906cfa8f0a6/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.084203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3c079-f1e9-4c28-919f-c0a42e457c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff2d50730e24f3b9983e44416421b05, user_id=3e80908e6ccb48ec984fa11268437eaa, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.088517 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3c079-f1e9-4c28-919f-c0a42e457c87 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.094256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75c7d8-73c9-45d0-90be-b9de105e205e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba0ddfd3efd41ee9687efe014c40bf0, user_id=0267350339c44000bcc49906cfa8f0a6, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.097731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75c7d8-73c9-45d0-90be-b9de105e205e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.114406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 804/1610] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:38 2022] PUT /identity/v3/projects/8ff2d50730e24f3b9983e44416421b05/users/3e80908e6ccb48ec984fa11268437eaa/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:39.115355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3c079-f1e9-4c28-919f-c0a42e457c87 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.117673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 807/1611] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:38 2022] PUT /identity/v3/projects/0ba0ddfd3efd41ee9687efe014c40bf0/users/0267350339c44000bcc49906cfa8f0a6/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:39.119252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5739-dbb0-4b22-96fa-d431435b0a41 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.119543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5739-dbb0-4b22-96fa-d431435b0a41 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.119822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5739-dbb0-4b22-96fa-d431435b0a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.153026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9c7b5739-dbb0-4b22-96fa-d431435b0a41 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:39.155109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9c7b5739-dbb0-4b22-96fa-d431435b0a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:39.200355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:39.218568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3c079-f1e9-4c28-919f-c0a42e457c87 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.254195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 808/1612] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:39.255476 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b5739-dbb0-4b22-96fa-d431435b0a41 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.267192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b5739-dbb0-4b22-96fa-d431435b0a41 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.273320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc6ae4-d4ed-4f79-959f-7fa41df0edc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.274360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc6ae4-d4ed-4f79-959f-7fa41df0edc6 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.274822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc6ae4-d4ed-4f79-959f-7fa41df0edc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.275250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc6ae4-d4ed-4f79-959f-7fa41df0edc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.276292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc6ae4-d4ed-4f79-959f-7fa41df0edc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.279154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc6ae4-d4ed-4f79-959f-7fa41df0edc6 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.281907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a9552-b743-420e-842b-89d372592a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.282569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a9552-b743-420e-842b-89d372592a22 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.282852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a9552-b743-420e-842b-89d372592a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.283260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a9552-b743-420e-842b-89d372592a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.291818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 809/1613] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:39.293282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc6ae4-d4ed-4f79-959f-7fa41df0edc6 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.294167 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a9552-b743-420e-842b-89d372592a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.296260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a9552-b743-420e-842b-89d372592a22 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.306021 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc6ae4-d4ed-4f79-959f-7fa41df0edc6 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.308328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 805/1614] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:39.309396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a9552-b743-420e-842b-89d372592a22 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.312236 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5c554-db10-4073-bd9d-8bd55e2e15b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.313316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5c554-db10-4073-bd9d-8bd55e2e15b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.313912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5c554-db10-4073-bd9d-8bd55e2e15b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.314404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5c554-db10-4073-bd9d-8bd55e2e15b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.315337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5c554-db10-4073-bd9d-8bd55e2e15b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.317534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5c554-db10-4073-bd9d-8bd55e2e15b4 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.319752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a9552-b743-420e-842b-89d372592a22 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.325907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b04c69-cd55-4fa9-bf5a-042fc65a8967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.326723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b04c69-cd55-4fa9-bf5a-042fc65a8967 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.327313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b04c69-cd55-4fa9-bf5a-042fc65a8967 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.327313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b04c69-cd55-4fa9-bf5a-042fc65a8967 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.327942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b04c69-cd55-4fa9-bf5a-042fc65a8967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.330137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b04c69-cd55-4fa9-bf5a-042fc65a8967 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.354494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 810/1615] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:39 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:39.356517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3271bf6-7d64-4bf7-a34c-8fde68c5de89 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.356968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3271bf6-7d64-4bf7-a34c-8fde68c5de89 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.358000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3271bf6-7d64-4bf7-a34c-8fde68c5de89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.364216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 806/1616] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:39 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:39.365920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98896d-a2cb-4cf8-bf7f-a9166e487a02 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.366375 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98896d-a2cb-4cf8-bf7f-a9166e487a02 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.366677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98896d-a2cb-4cf8-bf7f-a9166e487a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.382616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c3271bf6-7d64-4bf7-a34c-8fde68c5de89 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:39.384482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c3271bf6-7d64-4bf7-a34c-8fde68c5de89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:39.400318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:39.414129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6f98896d-a2cb-4cf8-bf7f-a9166e487a02 None None] MFA Rules not processed for user `3e80908e6ccb48ec984fa11268437eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:39.416465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6f98896d-a2cb-4cf8-bf7f-a9166e487a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:39.494582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 811/1617] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:39.494582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3271bf6-7d64-4bf7-a34c-8fde68c5de89 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.494582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c3271bf6-7d64-4bf7-a34c-8fde68c5de89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:39.566566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 807/1618] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:39.567632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98896d-a2cb-4cf8-bf7f-a9166e487a02 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.583543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98896d-a2cb-4cf8-bf7f-a9166e487a02 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.600583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245f6d4-16e4-4f65-9e83-1f6ba1a941d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.601505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245f6d4-16e4-4f65-9e83-1f6ba1a941d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.601834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245f6d4-16e4-4f65-9e83-1f6ba1a941d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.602117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245f6d4-16e4-4f65-9e83-1f6ba1a941d6 admin admin] PATH_INFO: `/v3/projects/6d8b1593aab64207929471dcb695c0d5/users/646afa0e8f0149c1bcd95660fa1e3ebb/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.603134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3271bf6-7d64-4bf7-a34c-8fde68c5de89 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.618637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fd72e-fcc5-4e97-a1d3-55c80d867dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.619323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fd72e-fcc5-4e97-a1d3-55c80d867dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.619609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fd72e-fcc5-4e97-a1d3-55c80d867dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.619877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fd72e-fcc5-4e97-a1d3-55c80d867dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.637261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f245f6d4-16e4-4f65-9e83-1f6ba1a941d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8b1593aab64207929471dcb695c0d5, user_id=646afa0e8f0149c1bcd95660fa1e3ebb, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.640469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fd72e-fcc5-4e97-a1d3-55c80d867dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.642741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fd72e-fcc5-4e97-a1d3-55c80d867dc1 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.644289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f245f6d4-16e4-4f65-9e83-1f6ba1a941d6 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.664318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 812/1620] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:39.666122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fd72e-fcc5-4e97-a1d3-55c80d867dc1 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.672898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 808/1620] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:39 2022] PUT /identity/v3/projects/6d8b1593aab64207929471dcb695c0d5/users/646afa0e8f0149c1bcd95660fa1e3ebb/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:39.675869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245f6d4-16e4-4f65-9e83-1f6ba1a941d6 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.684562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fd72e-fcc5-4e97-a1d3-55c80d867dc1 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.738661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:39.759384 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245f6d4-16e4-4f65-9e83-1f6ba1a941d6 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.767956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00265b03-e75a-4297-9fb0-ac7253fcd00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.768653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00265b03-e75a-4297-9fb0-ac7253fcd00d admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.769002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00265b03-e75a-4297-9fb0-ac7253fcd00d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.769373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00265b03-e75a-4297-9fb0-ac7253fcd00d admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.770309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00265b03-e75a-4297-9fb0-ac7253fcd00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.772609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00265b03-e75a-4297-9fb0-ac7253fcd00d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.813962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 809/1621] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:39 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:39.815301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00265b03-e75a-4297-9fb0-ac7253fcd00d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.828426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2506-c0c7-4205-adff-43ce6cda87c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.829372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2506-c0c7-4205-adff-43ce6cda87c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.830034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2506-c0c7-4205-adff-43ce6cda87c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.830495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2506-c0c7-4205-adff-43ce6cda87c9 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.831357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2506-c0c7-4205-adff-43ce6cda87c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.833445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2506-c0c7-4205-adff-43ce6cda87c9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.868595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 813/1622] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:39 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:39.871533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e719f6a-4ad7-4ac5-b374-eb40ba82e421 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.872126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e719f6a-4ad7-4ac5-b374-eb40ba82e421 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.872560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e719f6a-4ad7-4ac5-b374-eb40ba82e421 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.874825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 814/1623] 10.222.0.66 () {54 vars in 961 bytes} [Fri Jul 1 19:52:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:52:39.876140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e719f6a-4ad7-4ac5-b374-eb40ba82e421 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.892890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e719f6a-4ad7-4ac5-b374-eb40ba82e421 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.899933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558912f-0f11-4b43-9feb-fc9108a7dc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.900998 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558912f-0f11-4b43-9feb-fc9108a7dc12 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.902159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558912f-0f11-4b43-9feb-fc9108a7dc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.902646 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558912f-0f11-4b43-9feb-fc9108a7dc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.903573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00265b03-e75a-4297-9fb0-ac7253fcd00d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.904212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558912f-0f11-4b43-9feb-fc9108a7dc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.906801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558912f-0f11-4b43-9feb-fc9108a7dc12 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.910253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dda114-c424-4c56-849b-74116515cfc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.913619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dda114-c424-4c56-849b-74116515cfc3 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.914097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dda114-c424-4c56-849b-74116515cfc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.914949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dda114-c424-4c56-849b-74116515cfc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.922955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 815/1624] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:39.924063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558912f-0f11-4b43-9feb-fc9108a7dc12 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.930377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dda114-c424-4c56-849b-74116515cfc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.932181 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dda114-c424-4c56-849b-74116515cfc3 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.939621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558912f-0f11-4b43-9feb-fc9108a7dc12 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.944356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 810/1625] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:39.945217 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dda114-c424-4c56-849b-74116515cfc3 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.946293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30094e-0990-47d0-be96-68c303d862f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.947157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30094e-0990-47d0-be96-68c303d862f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.947792 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30094e-0990-47d0-be96-68c303d862f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.948911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30094e-0990-47d0-be96-68c303d862f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.949862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30094e-0990-47d0-be96-68c303d862f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.953519 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30094e-0990-47d0-be96-68c303d862f2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.957408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dda114-c424-4c56-849b-74116515cfc3 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:39.963345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7448e-cfcb-4ee5-8b31-b117dc4191a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:39.964783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7448e-cfcb-4ee5-8b31-b117dc4191a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.965195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7448e-cfcb-4ee5-8b31-b117dc4191a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.965648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7448e-cfcb-4ee5-8b31-b117dc4191a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.966502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7448e-cfcb-4ee5-8b31-b117dc4191a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:39.968321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7448e-cfcb-4ee5-8b31-b117dc4191a8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:39.970856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 816/1626] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:39.972683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0cae3-b66f-4bdd-bc51-ff77dad6ebd8 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:39.973137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0cae3-b66f-4bdd-bc51-ff77dad6ebd8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:39.973899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0cae3-b66f-4bdd-bc51-ff77dad6ebd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:39.989342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 811/1627] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:39.990724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7448e-cfcb-4ee5-8b31-b117dc4191a8 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:39.997482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7db0cae3-b66f-4bdd-bc51-ff77dad6ebd8 None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:39.999070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7db0cae3-b66f-4bdd-bc51-ff77dad6ebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:40.004635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7448e-cfcb-4ee5-8b31-b117dc4191a8 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:40.009901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b39d2-0bc5-4493-a1fd-585f1b405f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:40.011844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b39d2-0bc5-4493-a1fd-585f1b405f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:40.012464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b39d2-0bc5-4493-a1fd-585f1b405f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:40.013037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b39d2-0bc5-4493-a1fd-585f1b405f5a admin admin] PATH_INFO: `/v3/projects/6d8b1593aab64207929471dcb695c0d5/users/646afa0e8f0149c1bcd95660fa1e3ebb/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:40.016176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b39d2-0bc5-4493-a1fd-585f1b405f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8b1593aab64207929471dcb695c0d5, user_id=646afa0e8f0149c1bcd95660fa1e3ebb, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:40.019087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b39d2-0bc5-4493-a1fd-585f1b405f5a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:40.036920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 812/1628] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:39 2022] PUT /identity/v3/projects/6d8b1593aab64207929471dcb695c0d5/users/646afa0e8f0149c1bcd95660fa1e3ebb/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:40.037797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b39d2-0bc5-4493-a1fd-585f1b405f5a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:40.043670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:40.209863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 817/1629] 10.222.0.66 () {58 vars in 1058 bytes} [Fri Jul 1 19:52:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:40.209863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0cae3-b66f-4bdd-bc51-ff77dad6ebd8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:40.214311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b39d2-0bc5-4493-a1fd-585f1b405f5a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:40.214811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0cae3-b66f-4bdd-bc51-ff77dad6ebd8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:40.215315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cb288-79b6-420f-a4bb-211ba043f830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:40.215315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cb288-79b6-420f-a4bb-211ba043f830 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:40.215315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cb288-79b6-420f-a4bb-211ba043f830 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:40.215315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cb288-79b6-420f-a4bb-211ba043f830 admin admin] PATH_INFO: `/v3/projects/90153183d10f495486fe9a7bd35283fc/users/b2eb8c198041421f8459daf6c357e9e8/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:40.216014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44814272-6beb-4d96-b564-6f43d369fe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:40.216014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44814272-6beb-4d96-b564-6f43d369fe7c admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:40.216014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44814272-6beb-4d96-b564-6f43d369fe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:40.216014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44814272-6beb-4d96-b564-6f43d369fe7c admin admin] PATH_INFO: `/v3/projects/405882e26cd7472bb559103e5f27e9b3/users/e7f9a079a28948fc9b0e9e71fbf66645/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:40.227355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44814272-6beb-4d96-b564-6f43d369fe7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=405882e26cd7472bb559103e5f27e9b3, user_id=e7f9a079a28948fc9b0e9e71fbf66645, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:40.230587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44814272-6beb-4d96-b564-6f43d369fe7c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:40.232245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cb288-79b6-420f-a4bb-211ba043f830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90153183d10f495486fe9a7bd35283fc, user_id=b2eb8c198041421f8459daf6c357e9e8, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:40.240697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cb288-79b6-420f-a4bb-211ba043f830 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:40.253360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 818/1630] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:40 2022] PUT /identity/v3/projects/405882e26cd7472bb559103e5f27e9b3/users/e7f9a079a28948fc9b0e9e71fbf66645/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:40.254727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f166a-2149-4820-aaaf-0e0dfb7e02d6 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:40.255010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f166a-2149-4820-aaaf-0e0dfb7e02d6 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:40.255284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f166a-2149-4820-aaaf-0e0dfb7e02d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:40.279358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 813/1631] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:40 2022] PUT /identity/v3/projects/90153183d10f495486fe9a7bd35283fc/users/b2eb8c198041421f8459daf6c357e9e8/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:40.280417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cb288-79b6-420f-a4bb-211ba043f830 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:40.319805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-525f166a-2149-4820-aaaf-0e0dfb7e02d6 None None] MFA Rules not processed for user `646afa0e8f0149c1bcd95660fa1e3ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:40.322491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-525f166a-2149-4820-aaaf-0e0dfb7e02d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:40.353380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:40.404582 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cb288-79b6-420f-a4bb-211ba043f830 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:40.420668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60d96d-446e-485c-9e18-dda186850176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:40.421399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60d96d-446e-485c-9e18-dda186850176 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:40.421728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60d96d-446e-485c-9e18-dda186850176 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:40.422010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60d96d-446e-485c-9e18-dda186850176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:40.453481 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 819/1632] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:40.454392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f166a-2149-4820-aaaf-0e0dfb7e02d6 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:40.545656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60d96d-446e-485c-9e18-dda186850176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:40.552977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60d96d-446e-485c-9e18-dda186850176 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:40.576203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 814/1633] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:40.577909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60d96d-446e-485c-9e18-dda186850176 service nova] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:40.608680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f166a-2149-4820-aaaf-0e0dfb7e02d6 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:40.620450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba459c3-e66a-44e6-ae30-aafab8351727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:40.621299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba459c3-e66a-44e6-ae30-aafab8351727 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:40.621594 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba459c3-e66a-44e6-ae30-aafab8351727 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:40.624846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba459c3-e66a-44e6-ae30-aafab8351727 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:40.625922 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba459c3-e66a-44e6-ae30-aafab8351727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:40.634887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba459c3-e66a-44e6-ae30-aafab8351727 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:40.679050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 820/1634] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:40.686684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba459c3-e66a-44e6-ae30-aafab8351727 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:40.733341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:40.761844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60d96d-446e-485c-9e18-dda186850176 service nova] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:40.777652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956cbe2-e516-426e-bc89-77677b70e434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:40.778604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956cbe2-e516-426e-bc89-77677b70e434 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:40.778875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956cbe2-e516-426e-bc89-77677b70e434 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:40.779150 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956cbe2-e516-426e-bc89-77677b70e434 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:40.780188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956cbe2-e516-426e-bc89-77677b70e434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:40.781529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956cbe2-e516-426e-bc89-77677b70e434 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:40.820504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 815/1635] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:40.820504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956cbe2-e516-426e-bc89-77677b70e434 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:40.844284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba459c3-e66a-44e6-ae30-aafab8351727 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:40.851072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956cbe2-e516-426e-bc89-77677b70e434 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:40.861775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4937aa-d0b9-4cb2-8df1-8f71d74f3748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:40.862454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4937aa-d0b9-4cb2-8df1-8f71d74f3748 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:40.862734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4937aa-d0b9-4cb2-8df1-8f71d74f3748 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:40.863008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4937aa-d0b9-4cb2-8df1-8f71d74f3748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:40.867790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964f2f4-7d31-4c77-9851-2487a6737e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:40.869438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964f2f4-7d31-4c77-9851-2487a6737e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:40.869757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964f2f4-7d31-4c77-9851-2487a6737e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:40.870143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964f2f4-7d31-4c77-9851-2487a6737e17 admin admin] PATH_INFO: `/v3/projects/405882e26cd7472bb559103e5f27e9b3/users/e7f9a079a28948fc9b0e9e71fbf66645/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:40.879185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964f2f4-7d31-4c77-9851-2487a6737e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=405882e26cd7472bb559103e5f27e9b3, user_id=e7f9a079a28948fc9b0e9e71fbf66645, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:40.887673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964f2f4-7d31-4c77-9851-2487a6737e17 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:40.894041 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4937aa-d0b9-4cb2-8df1-8f71d74f3748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:40.897135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4937aa-d0b9-4cb2-8df1-8f71d74f3748 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:40.937900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 816/1636] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:40 2022] PUT /identity/v3/projects/405882e26cd7472bb559103e5f27e9b3/users/e7f9a079a28948fc9b0e9e71fbf66645/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:40.939039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964f2f4-7d31-4c77-9851-2487a6737e17 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:40.939957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 821/1637] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:40.941244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5151ee-9772-4f6d-ae1b-bfe1f9b9a86b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:40.941528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5151ee-9772-4f6d-ae1b-bfe1f9b9a86b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:40.941834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5151ee-9772-4f6d-ae1b-bfe1f9b9a86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:41.028358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-aa5151ee-9772-4f6d-ae1b-bfe1f9b9a86b None None] MFA Rules not processed for user `e7f9a079a28948fc9b0e9e71fbf66645`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:41.035425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-aa5151ee-9772-4f6d-ae1b-bfe1f9b9a86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:41.129814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:41.143337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964f2f4-7d31-4c77-9851-2487a6737e17 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:41.156907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e16ac6-ad0e-4b94-9c3c-1b5b7f149213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:41.157428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e16ac6-ad0e-4b94-9c3c-1b5b7f149213 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:41.162471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e16ac6-ad0e-4b94-9c3c-1b5b7f149213 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:41.162471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e16ac6-ad0e-4b94-9c3c-1b5b7f149213 admin admin] PATH_INFO: `/v3/projects/90153183d10f495486fe9a7bd35283fc/users/b2eb8c198041421f8459daf6c357e9e8/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:41.166101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e16ac6-ad0e-4b94-9c3c-1b5b7f149213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90153183d10f495486fe9a7bd35283fc, user_id=b2eb8c198041421f8459daf6c357e9e8, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:41.170272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e16ac6-ad0e-4b94-9c3c-1b5b7f149213 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:41.205518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 817/1638] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:40 2022] PUT /identity/v3/projects/90153183d10f495486fe9a7bd35283fc/users/b2eb8c198041421f8459daf6c357e9e8/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:41.206459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e16ac6-ad0e-4b94-9c3c-1b5b7f149213 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:41.222213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 822/1639] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:40 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:41.224259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc63857-2f4c-48d1-a56f-0dae0d36240c None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:41.224606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc63857-2f4c-48d1-a56f-0dae0d36240c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:41.226401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc63857-2f4c-48d1-a56f-0dae0d36240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:41.275726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-8bc63857-2f4c-48d1-a56f-0dae0d36240c None None] MFA Rules not processed for user `b2eb8c198041421f8459daf6c357e9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:41.285410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8bc63857-2f4c-48d1-a56f-0dae0d36240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:41.425699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 823/1640] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:41.425699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc63857-2f4c-48d1-a56f-0dae0d36240c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:41.527658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:41.532086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e16ac6-ad0e-4b94-9c3c-1b5b7f149213 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:41.537879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dce91c-9a48-46c6-8db7-740efdfff35b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:41.539068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce91c-9a48-46c6-8db7-740efdfff35b service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:41.539360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce91c-9a48-46c6-8db7-740efdfff35b service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:41.539729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce91c-9a48-46c6-8db7-740efdfff35b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:41.566014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc63857-2f4c-48d1-a56f-0dae0d36240c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:41.570139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38264c-be48-433b-a719-6dae75639295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:41.570799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38264c-be48-433b-a719-6dae75639295 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:41.571083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38264c-be48-433b-a719-6dae75639295 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:41.571506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38264c-be48-433b-a719-6dae75639295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:41.607695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dce91c-9a48-46c6-8db7-740efdfff35b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:41.609180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dce91c-9a48-46c6-8db7-740efdfff35b service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:41.619523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 818/1641] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:41 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:41.649160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38264c-be48-433b-a719-6dae75639295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:41.650745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38264c-be48-433b-a719-6dae75639295 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:41.664921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 824/1642] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:41 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:41.716902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dce91c-9a48-46c6-8db7-740efdfff35b service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:41.731507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dce91c-9a48-46c6-8db7-740efdfff35b service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:41.740053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7586e-fb76-4694-8ab7-6fc277e3a666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:41.741638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7586e-fb76-4694-8ab7-6fc277e3a666 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:41.742246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7586e-fb76-4694-8ab7-6fc277e3a666 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:41.742800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7586e-fb76-4694-8ab7-6fc277e3a666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:41.761737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7586e-fb76-4694-8ab7-6fc277e3a666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:41.763646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7586e-fb76-4694-8ab7-6fc277e3a666 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:41.783486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 819/1643] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:41.908044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1d0c-4030-4ee6-ac1f-16d698970fec None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:41.908342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1d0c-4030-4ee6-ac1f-16d698970fec None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:41.908627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1d0c-4030-4ee6-ac1f-16d698970fec None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:41.910716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 825/1644] 10.222.0.66 () {54 vars in 961 bytes} [Fri Jul 1 19:52:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:52:41.923921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a34e4-4b50-4861-b4fa-04757478403d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:41.924172 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a34e4-4b50-4861-b4fa-04757478403d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:41.925435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a34e4-4b50-4861-b4fa-04757478403d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:41.967342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-158a34e4-4b50-4861-b4fa-04757478403d None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:41.973560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-158a34e4-4b50-4861-b4fa-04757478403d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:42.145475 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 820/1645] 10.222.0.66 () {58 vars in 1058 bytes} [Fri Jul 1 19:52:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:42.151891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffeba77-b99b-4b7d-a3a2-9685b4e820f8 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:42.152266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffeba77-b99b-4b7d-a3a2-9685b4e820f8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:42.152652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffeba77-b99b-4b7d-a3a2-9685b4e820f8 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:42.156358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 826/1646] 10.222.0.66 () {54 vars in 912 bytes} [Fri Jul 1 19:52:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:52:42.162674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467d05-8963-48b6-ad9c-8b154fcd829e None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:42.163259 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467d05-8963-48b6-ad9c-8b154fcd829e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:42.163641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467d05-8963-48b6-ad9c-8b154fcd829e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:42.219908 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffeba77-b99b-4b7d-a3a2-9685b4e820f8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:42.225993 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-98467d05-8963-48b6-ad9c-8b154fcd829e None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:42.227222 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-98467d05-8963-48b6-ad9c-8b154fcd829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:42.240622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffeba77-b99b-4b7d-a3a2-9685b4e820f8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:42.250960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 821/1647] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:52:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:42.268559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792e539-9d4b-45b3-83c3-06e5ebb7b6ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:42.270980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792e539-9d4b-45b3-83c3-06e5ebb7b6ca service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:42.271418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792e539-9d4b-45b3-83c3-06e5ebb7b6ca service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:42.271588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792e539-9d4b-45b3-83c3-06e5ebb7b6ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:42.321871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467d05-8963-48b6-ad9c-8b154fcd829e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:42.337762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467d05-8963-48b6-ad9c-8b154fcd829e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:42.347403 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9167388-d46b-4acf-ba4b-ac8a398f8a07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:42.348065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9167388-d46b-4acf-ba4b-ac8a398f8a07 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:42.348342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9167388-d46b-4acf-ba4b-ac8a398f8a07 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:42.348620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9167388-d46b-4acf-ba4b-ac8a398f8a07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:42.383956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9167388-d46b-4acf-ba4b-ac8a398f8a07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:42.386586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9167388-d46b-4acf-ba4b-ac8a398f8a07 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:42.394172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792e539-9d4b-45b3-83c3-06e5ebb7b6ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:42.397310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792e539-9d4b-45b3-83c3-06e5ebb7b6ca service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:42.399406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 822/1648] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:52:42.413087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 827/1649] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:52:42.736764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3686a6-b755-4a49-b088-a73d96b8ccb9 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:42.737371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3686a6-b755-4a49-b088-a73d96b8ccb9 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:42.737371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3686a6-b755-4a49-b088-a73d96b8ccb9 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:42.738917 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 823/1650] 10.222.0.66 () {54 vars in 912 bytes} [Fri Jul 1 19:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:52:42.749843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ba725-5b91-4bfa-9b0b-29b020d6a8a5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:42.750366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ba725-5b91-4bfa-9b0b-29b020d6a8a5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:42.750668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ba725-5b91-4bfa-9b0b-29b020d6a8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:42.793002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-932ba725-5b91-4bfa-9b0b-29b020d6a8a5 None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:42.796138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-932ba725-5b91-4bfa-9b0b-29b020d6a8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:42.858141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 828/1651] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:52:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:42.880671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3686a6-b755-4a49-b088-a73d96b8ccb9 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:42.889620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:42.908019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3686a6-b755-4a49-b088-a73d96b8ccb9 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:42.913225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae72bc-0656-4aef-acd2-dbe6885e4186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:42.914041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae72bc-0656-4aef-acd2-dbe6885e4186 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:42.914333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae72bc-0656-4aef-acd2-dbe6885e4186 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:42.914608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae72bc-0656-4aef-acd2-dbe6885e4186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:42.954002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae72bc-0656-4aef-acd2-dbe6885e4186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:42.955460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae72bc-0656-4aef-acd2-dbe6885e4186 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:42.974973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 824/1652] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:43.118324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ba725-5b91-4bfa-9b0b-29b020d6a8a5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:43.256961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ba725-5b91-4bfa-9b0b-29b020d6a8a5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:43.275927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:43.282676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:43.283628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:43.288285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] PATH_INFO: `/v3/users/3e80908e6ccb48ec984fa11268437eaa` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:43.290976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e80908e6ccb48ec984fa11268437eaa)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:43.297159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:43.342303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:43.420374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e80908e6ccb48ec984fa11268437eaa'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:43.457293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e80908e6ccb48ec984fa11268437eaa'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:43.503670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e80908e6ccb48ec984fa11268437eaa'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:43.516197 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e80908e6ccb48ec984fa11268437eaa'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:43.518198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e80908e6ccb48ec984fa11268437eaa'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:43.536531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 829/1653] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:52:43 2022] DELETE /identity/v3/users/3e80908e6ccb48ec984fa11268437eaa => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:43.541171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9f7bf-b82b-4492-a2af-bfd5d8aec59b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:43.541457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9f7bf-b82b-4492-a2af-bfd5d8aec59b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:43.542430 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9f7bf-b82b-4492-a2af-bfd5d8aec59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:43.598260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7bc9f7bf-b82b-4492-a2af-bfd5d8aec59b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:43.600674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7bc9f7bf-b82b-4492-a2af-bfd5d8aec59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:43.780204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 825/1654] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:43.798816 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:43.800778 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:43.992664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc5bec-dcca-40b5-90e2-814e8b2e877d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:43.992664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa023fb7-a3d3-4268-944d-d385f2fac38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:43.992664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa023fb7-a3d3-4268-944d-d385f2fac38e service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:43.992664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa023fb7-a3d3-4268-944d-d385f2fac38e service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:43.992979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa023fb7-a3d3-4268-944d-d385f2fac38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:44.033933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa023fb7-a3d3-4268-944d-d385f2fac38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:44.035454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa023fb7-a3d3-4268-944d-d385f2fac38e service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:44.053886 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 830/1655] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:45.865773 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9f7bf-b82b-4492-a2af-bfd5d8aec59b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:45.869610 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7bc9f7bf-b82b-4492-a2af-bfd5d8aec59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:45.914262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9f7bf-b82b-4492-a2af-bfd5d8aec59b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:45.921683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:45.924161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:45.924684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:45.925159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] PATH_INFO: `/v3/users/67319ef6317d4c8cac2cde40d9fa9625` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:45.927100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67319ef6317d4c8cac2cde40d9fa9625)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:45.929328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: WARNING py.warnings [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:52:45.929328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: warnings.warn(msg) Jul 01 19:52:45.929328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: Jul 01 19:52:45.931059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:46.078476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67319ef6317d4c8cac2cde40d9fa9625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:46.103319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67319ef6317d4c8cac2cde40d9fa9625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:46.104721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67319ef6317d4c8cac2cde40d9fa9625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:46.119029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa023fb7-a3d3-4268-944d-d385f2fac38e service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:46.121330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-aa023fb7-a3d3-4268-944d-d385f2fac38e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:46.151632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa023fb7-a3d3-4268-944d-d385f2fac38e service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:46.162371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67319ef6317d4c8cac2cde40d9fa9625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:46.163202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-bea36466-0745-47fa-a434-84cfffad8354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67319ef6317d4c8cac2cde40d9fa9625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:46.168432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:46.170847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:46.176166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:46.180845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] PATH_INFO: `/v3/projects/8ff2d50730e24f3b9983e44416421b05` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:46.184090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff2d50730e24f3b9983e44416421b05)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:46.190282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: WARNING py.warnings [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 19:52:46.190282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: warnings.warn(msg) Jul 01 19:52:46.190282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: Jul 01 19:52:46.191844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:46.222221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 826/1656] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:52:45 2022] DELETE /identity/v3/users/67319ef6317d4c8cac2cde40d9fa9625 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:46.224713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d330b-03dc-4af2-ae59-8e3a0fca4997 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:46.225836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d330b-03dc-4af2-ae59-8e3a0fca4997 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:46.226441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d330b-03dc-4af2-ae59-8e3a0fca4997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:46.253693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-880d330b-03dc-4af2-ae59-8e3a0fca4997 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:46.256259 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-880d330b-03dc-4af2-ae59-8e3a0fca4997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:46.360143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff2d50730e24f3b9983e44416421b05', 'request_id': 'req-93b527b4-29a0-45b7-9d7d-099d06942bb6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:46.361094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 827/1657] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:46.383780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff2d50730e24f3b9983e44416421b05', 'request_id': 'req-93b527b4-29a0-45b7-9d7d-099d06942bb6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:46.403282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff2d50730e24f3b9983e44416421b05', 'request_id': 'req-93b527b4-29a0-45b7-9d7d-099d06942bb6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:46.476540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] The token cache is being invalidate because project 8ff2d50730e24f3b9983e44416421b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:52:46.476840 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:46.478607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:46.487228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 831/1658] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:52:46 2022] DELETE /identity/v3/projects/8ff2d50730e24f3b9983e44416421b05 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:46.492878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae285d65-92d4-402d-aeb0-78593111c08d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:46.493241 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae285d65-92d4-402d-aeb0-78593111c08d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:46.493455 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae285d65-92d4-402d-aeb0-78593111c08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:46.516479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:46.545162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ae285d65-92d4-402d-aeb0-78593111c08d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:46.552278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ae285d65-92d4-402d-aeb0-78593111c08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:46.591206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:46.593372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:46.653539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:46.721231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 828/1659] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:46.722972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26682260-c1e5-4a9b-b338-f5453c283535 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:46.723264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26682260-c1e5-4a9b-b338-f5453c283535 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:46.723542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26682260-c1e5-4a9b-b338-f5453c283535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:46.766858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-26682260-c1e5-4a9b-b338-f5453c283535 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:46.774835 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-26682260-c1e5-4a9b-b338-f5453c283535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:46.778794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b527b4-29a0-45b7-9d7d-099d06942bb6 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:46.793153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af65943-db95-4534-840b-ee88e049ae12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:46.794301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af65943-db95-4534-840b-ee88e049ae12 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:46.794698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af65943-db95-4534-840b-ee88e049ae12 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:46.795080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af65943-db95-4534-840b-ee88e049ae12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:46.799110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 829/1660] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:46.802850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8af65943-db95-4534-840b-ee88e049ae12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:46.808050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26682260-c1e5-4a9b-b338-f5453c283535 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:46.826649 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26682260-c1e5-4a9b-b338-f5453c283535 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:46.841129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea714e-a099-473e-99a9-6d04ac39baca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:46.841557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea714e-a099-473e-99a9-6d04ac39baca admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:46.841868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea714e-a099-473e-99a9-6d04ac39baca admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:46.842143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea714e-a099-473e-99a9-6d04ac39baca admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:46.842894 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea714e-a099-473e-99a9-6d04ac39baca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:46.843179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea714e-a099-473e-99a9-6d04ac39baca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:46.844828 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea714e-a099-473e-99a9-6d04ac39baca admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:46.853854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af65943-db95-4534-840b-ee88e049ae12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:46.856450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af65943-db95-4534-840b-ee88e049ae12 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:46.892338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 830/1661] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:46.898782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 832/1662] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:46.898917 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea714e-a099-473e-99a9-6d04ac39baca admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:46.914575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea714e-a099-473e-99a9-6d04ac39baca admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:46.927706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc98f89-157a-45c4-b43c-2821d2022523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:46.928396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98f89-157a-45c4-b43c-2821d2022523 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:46.928678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98f89-157a-45c4-b43c-2821d2022523 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:46.928955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc98f89-157a-45c4-b43c-2821d2022523 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:46.929678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc98f89-157a-45c4-b43c-2821d2022523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:46.931749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc98f89-157a-45c4-b43c-2821d2022523 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:46.992923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 831/1663] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:46 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:46.998282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af65943-db95-4534-840b-ee88e049ae12 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:47.013085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af65943-db95-4534-840b-ee88e049ae12 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:47.247665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86aae39-8bfc-4b74-96c0-4e71c7e1e527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:47.249263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86aae39-8bfc-4b74-96c0-4e71c7e1e527 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:47.249551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86aae39-8bfc-4b74-96c0-4e71c7e1e527 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:47.249846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86aae39-8bfc-4b74-96c0-4e71c7e1e527 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:47.250649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86aae39-8bfc-4b74-96c0-4e71c7e1e527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:47.255325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86aae39-8bfc-4b74-96c0-4e71c7e1e527 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:47.312340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 833/1664] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:46 2022] POST /identity/v3/users => generated 303 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:47.315336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc98f89-157a-45c4-b43c-2821d2022523 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:47.331404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc98f89-157a-45c4-b43c-2821d2022523 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:47.346477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97240e8f-6967-498c-b29e-d17392de878e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:47.347286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240e8f-6967-498c-b29e-d17392de878e admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:47.347674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240e8f-6967-498c-b29e-d17392de878e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:47.348208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240e8f-6967-498c-b29e-d17392de878e admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:47.349059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97240e8f-6967-498c-b29e-d17392de878e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:47.351997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97240e8f-6967-498c-b29e-d17392de878e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:47.374217 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 832/1665] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:47.376033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86aae39-8bfc-4b74-96c0-4e71c7e1e527 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:47.391852 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86aae39-8bfc-4b74-96c0-4e71c7e1e527 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:47.431907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea63ae-be86-48e0-9ae1-bd82f39832d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:47.434545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea63ae-be86-48e0-9ae1-bd82f39832d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:47.439407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea63ae-be86-48e0-9ae1-bd82f39832d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:47.439987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea63ae-be86-48e0-9ae1-bd82f39832d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:47.441256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea63ae-be86-48e0-9ae1-bd82f39832d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:47.446132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea63ae-be86-48e0-9ae1-bd82f39832d0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:47.516792 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 834/1666] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:47.520605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97240e8f-6967-498c-b29e-d17392de878e admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:47.546698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97240e8f-6967-498c-b29e-d17392de878e admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:47.559581 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3989fa-1915-4002-9a6e-62e72f9f1642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:47.561727 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3989fa-1915-4002-9a6e-62e72f9f1642 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:47.564171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3989fa-1915-4002-9a6e-62e72f9f1642 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:47.564616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3989fa-1915-4002-9a6e-62e72f9f1642 admin admin] PATH_INFO: `/v3/projects/4de3855f7cab4f1b828d5ab385e4f74e/users/b7e7cb7a344943c788572aa6b38d10c2/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:47.633131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3989fa-1915-4002-9a6e-62e72f9f1642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de3855f7cab4f1b828d5ab385e4f74e, user_id=b7e7cb7a344943c788572aa6b38d10c2, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:47.637843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3989fa-1915-4002-9a6e-62e72f9f1642 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:47.677635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 833/1667] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:47 2022] PUT /identity/v3/projects/4de3855f7cab4f1b828d5ab385e4f74e/users/b7e7cb7a344943c788572aa6b38d10c2/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:47.683549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e49bb-4452-4fc9-b996-f4845f7122db None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:47.688431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e49bb-4452-4fc9-b996-f4845f7122db None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:47.689441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e49bb-4452-4fc9-b996-f4845f7122db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:47.777404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f42e49bb-4452-4fc9-b996-f4845f7122db None None] MFA Rules not processed for user `b7e7cb7a344943c788572aa6b38d10c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:47.784873 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f42e49bb-4452-4fc9-b996-f4845f7122db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:48.040757 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 835/1668] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:47 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:48.047398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3989fa-1915-4002-9a6e-62e72f9f1642 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:48.226022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3989fa-1915-4002-9a6e-62e72f9f1642 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:48.241529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187b003-8872-46c6-9e2c-3f700160781a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:48.242261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187b003-8872-46c6-9e2c-3f700160781a admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:48.242547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187b003-8872-46c6-9e2c-3f700160781a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:48.242824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187b003-8872-46c6-9e2c-3f700160781a admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:48.245012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187b003-8872-46c6-9e2c-3f700160781a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:48.256451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187b003-8872-46c6-9e2c-3f700160781a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:48.304174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 834/1669] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:48 2022] POST /identity/v3/projects => generated 405 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:48.308064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e49bb-4452-4fc9-b996-f4845f7122db None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:48.330775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e49bb-4452-4fc9-b996-f4845f7122db None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:48.400087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:48.496084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32b1f8-b0a1-46db-9475-c9446f07dfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:48.497210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32b1f8-b0a1-46db-9475-c9446f07dfd9 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:48.497774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32b1f8-b0a1-46db-9475-c9446f07dfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:48.498276 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32b1f8-b0a1-46db-9475-c9446f07dfd9 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:48.499416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32b1f8-b0a1-46db-9475-c9446f07dfd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:48.502634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32b1f8-b0a1-46db-9475-c9446f07dfd9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:48.580126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 836/1670] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:48 2022] POST /identity/v3/users => generated 309 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:48.583307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187b003-8872-46c6-9e2c-3f700160781a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:48.608650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187b003-8872-46c6-9e2c-3f700160781a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:48.643323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fd478-c94c-4221-ad7b-0d2b87339c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:48.645834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fd478-c94c-4221-ad7b-0d2b87339c96 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:48.646281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fd478-c94c-4221-ad7b-0d2b87339c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:48.646733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fd478-c94c-4221-ad7b-0d2b87339c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:48.652598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fd478-c94c-4221-ad7b-0d2b87339c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:48.655115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fd478-c94c-4221-ad7b-0d2b87339c96 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:48.707406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 835/1671] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:48.708314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32b1f8-b0a1-46db-9475-c9446f07dfd9 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:48.725809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32b1f8-b0a1-46db-9475-c9446f07dfd9 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:48.746836 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ca21a-c1a9-4acf-b886-4834376245e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:48.749255 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ca21a-c1a9-4acf-b886-4834376245e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:48.750400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ca21a-c1a9-4acf-b886-4834376245e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:48.750920 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ca21a-c1a9-4acf-b886-4834376245e8 admin admin] PATH_INFO: `/v3/projects/388c31e0cd074222aac4acbebcdfd507/users/2199bee011fe412dbf04a9f919504ea3/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:48.849865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ca21a-c1a9-4acf-b886-4834376245e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=388c31e0cd074222aac4acbebcdfd507, user_id=2199bee011fe412dbf04a9f919504ea3, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:48.860492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ca21a-c1a9-4acf-b886-4834376245e8 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:48.917630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 837/1672] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:48 2022] PUT /identity/v3/projects/388c31e0cd074222aac4acbebcdfd507/users/2199bee011fe412dbf04a9f919504ea3/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:48.921422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fd478-c94c-4221-ad7b-0d2b87339c96 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:48.926908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:48.939523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fd478-c94c-4221-ad7b-0d2b87339c96 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:49.158919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb25568-ed81-4d0c-a11a-31575817608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:49.159650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb25568-ed81-4d0c-a11a-31575817608f admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:49.159935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb25568-ed81-4d0c-a11a-31575817608f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:49.160309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb25568-ed81-4d0c-a11a-31575817608f admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:49.165175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb25568-ed81-4d0c-a11a-31575817608f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:49.167464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb25568-ed81-4d0c-a11a-31575817608f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:49.195460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ca21a-c1a9-4acf-b886-4834376245e8 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:49.197437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fa9ca21a-c1a9-4acf-b886-4834376245e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:49.213524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 836/1673] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:49.234271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb25568-ed81-4d0c-a11a-31575817608f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:49.239614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ca21a-c1a9-4acf-b886-4834376245e8 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:49.262345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:49.263428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:49.264512 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:49.264919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] PATH_INFO: `/v3/projects/f15a8f44ad544e80982c570d6d12473b` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:49.272653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f15a8f44ad544e80982c570d6d12473b)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:49.276732 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:49.303536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb25568-ed81-4d0c-a11a-31575817608f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:49.321816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf7bac-343e-4718-bae8-c0e69d207480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:49.326388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf7bac-343e-4718-bae8-c0e69d207480 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:49.328592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf7bac-343e-4718-bae8-c0e69d207480 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:49.328885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf7bac-343e-4718-bae8-c0e69d207480 admin admin] PATH_INFO: `/v3/projects/388c31e0cd074222aac4acbebcdfd507/users/2199bee011fe412dbf04a9f919504ea3/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:49.333104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf7bac-343e-4718-bae8-c0e69d207480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=388c31e0cd074222aac4acbebcdfd507, user_id=2199bee011fe412dbf04a9f919504ea3, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:49.339032 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf7bac-343e-4718-bae8-c0e69d207480 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:49.367373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 837/1674] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:49 2022] PUT /identity/v3/projects/388c31e0cd074222aac4acbebcdfd507/users/2199bee011fe412dbf04a9f919504ea3/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:49.379620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6279d20-1d27-484b-a425-73fd17f70656 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:49.379912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6279d20-1d27-484b-a425-73fd17f70656 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:49.380197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6279d20-1d27-484b-a425-73fd17f70656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:49.416549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15a8f44ad544e80982c570d6d12473b', 'request_id': 'req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:49.435487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15a8f44ad544e80982c570d6d12473b', 'request_id': 'req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:49.444039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15a8f44ad544e80982c570d6d12473b', 'request_id': 'req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:49.444960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d6279d20-1d27-484b-a425-73fd17f70656 None None] MFA Rules not processed for user `2199bee011fe412dbf04a9f919504ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:49.450162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d6279d20-1d27-484b-a425-73fd17f70656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:49.461547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] The token cache is being invalidate because project f15a8f44ad544e80982c570d6d12473b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:52:49.462586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:49.463736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:49.469721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 838/1675] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:52:49 2022] DELETE /identity/v3/projects/f15a8f44ad544e80982c570d6d12473b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:52:49.470986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:49.473389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:49.667292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35ea86-e02b-4b77-ac79-a7a6e5855aae admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:49.694363 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:49.695697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:49.696901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:49.699059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] PATH_INFO: `/v3/users/e7f9a079a28948fc9b0e9e71fbf66645` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:49.702777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f9a079a28948fc9b0e9e71fbf66645)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:49.711826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:49.757225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 838/1676] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:49 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:52:49.771697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6279d20-1d27-484b-a425-73fd17f70656 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:49.774280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d6279d20-1d27-484b-a425-73fd17f70656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:49.799291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:49.871309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f9a079a28948fc9b0e9e71fbf66645'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:49.910376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f9a079a28948fc9b0e9e71fbf66645'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:49.929895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f9a079a28948fc9b0e9e71fbf66645'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:49.942453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f9a079a28948fc9b0e9e71fbf66645'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:49.943659 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f9a079a28948fc9b0e9e71fbf66645'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:49.954887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 839/1677] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:52:49 2022] DELETE /identity/v3/users/e7f9a079a28948fc9b0e9e71fbf66645 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:49.976408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6279d20-1d27-484b-a425-73fd17f70656 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:50.021047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:50.087169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a84328-8eec-451e-8086-f8f1d99c34ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:50.087914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84328-8eec-451e-8086-f8f1d99c34ed service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:50.088204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84328-8eec-451e-8086-f8f1d99c34ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:50.089192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84328-8eec-451e-8086-f8f1d99c34ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:50.092045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-42a84328-8eec-451e-8086-f8f1d99c34ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:50.199504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a84328-8eec-451e-8086-f8f1d99c34ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:50.201072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a84328-8eec-451e-8086-f8f1d99c34ed service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:50.233753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 839/1678] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:49 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:52.055807 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:52.059923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:52.174253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0df70-c2bb-48fb-86e2-47455cd3810c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:52.177142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610da3-dab9-4cce-b1e1-1c2c0c0920c2 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:52.178584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610da3-dab9-4cce-b1e1-1c2c0c0920c2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:52.179424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01610da3-dab9-4cce-b1e1-1c2c0c0920c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:52.180267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:52.181243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:52.182773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:52.185294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] PATH_INFO: `/v3/users/b7e7cb7a344943c788572aa6b38d10c2` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:52.186176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e7cb7a344943c788572aa6b38d10c2)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:52.188711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:52.278776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-01610da3-dab9-4cce-b1e1-1c2c0c0920c2 None None] MFA Rules not processed for user `3307be3159654f55be46e75c82584bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:52.280841 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e7cb7a344943c788572aa6b38d10c2'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:52.282803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-01610da3-dab9-4cce-b1e1-1c2c0c0920c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:52.313798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e7cb7a344943c788572aa6b38d10c2'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:52.342946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e7cb7a344943c788572aa6b38d10c2'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:52.370065 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e7cb7a344943c788572aa6b38d10c2'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:52.375232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ebe4c44e-1355-4a45-9388-435da482e679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e7cb7a344943c788572aa6b38d10c2'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:52.398117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 840/1679] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:52:52 2022] DELETE /identity/v3/users/b7e7cb7a344943c788572aa6b38d10c2 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:52.400057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06772e6-09fa-48ed-88af-e387be823049 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:52.400875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06772e6-09fa-48ed-88af-e387be823049 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:52.403517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06772e6-09fa-48ed-88af-e387be823049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:52.441285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-b06772e6-09fa-48ed-88af-e387be823049 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:52.445365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b06772e6-09fa-48ed-88af-e387be823049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:52.474131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:52.481164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 840/1680] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:52.483070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610da3-dab9-4cce-b1e1-1c2c0c0920c2 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:52.486419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-01610da3-dab9-4cce-b1e1-1c2c0c0920c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:52.518018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01610da3-dab9-4cce-b1e1-1c2c0c0920c2 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:52.595341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 841/1681] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:52.601286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06772e6-09fa-48ed-88af-e387be823049 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:52.605601 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b06772e6-09fa-48ed-88af-e387be823049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:52.672311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:52.673935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:52.674707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:52.675094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] PATH_INFO: `/v3/projects/405882e26cd7472bb559103e5f27e9b3` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:52.678232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] RBAC: Authorizing `identity:delete_project(project_id=405882e26cd7472bb559103e5f27e9b3)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:52.680830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:52.704400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:52.801177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06772e6-09fa-48ed-88af-e387be823049 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:52.816492 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '405882e26cd7472bb559103e5f27e9b3', 'request_id': 'req-9b10ddf5-07c3-46a2-be05-ceee7111645c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:52.843512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '405882e26cd7472bb559103e5f27e9b3', 'request_id': 'req-9b10ddf5-07c3-46a2-be05-ceee7111645c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:52.852997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '405882e26cd7472bb559103e5f27e9b3', 'request_id': 'req-9b10ddf5-07c3-46a2-be05-ceee7111645c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:52.917037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] The token cache is being invalidate because project 405882e26cd7472bb559103e5f27e9b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:52:52.917814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:52.923447 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:52.928254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 841/1682] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:52:52 2022] DELETE /identity/v3/projects/405882e26cd7472bb559103e5f27e9b3 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:52.929448 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:52.937769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:52.971111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8dcce-efeb-49fc-93bb-8f8da44d4072 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:52.971802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8dcce-efeb-49fc-93bb-8f8da44d4072 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:52.972086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8dcce-efeb-49fc-93bb-8f8da44d4072 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:52.972367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8dcce-efeb-49fc-93bb-8f8da44d4072 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:52.975162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4eb8dcce-efeb-49fc-93bb-8f8da44d4072 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:53.149105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:53.163499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10ddf5-07c3-46a2-be05-ceee7111645c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:53.173779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8dcce-efeb-49fc-93bb-8f8da44d4072 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:53.174915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ce22c-52da-41f3-8ec4-4f4e970d05c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:53.176104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce22c-52da-41f3-8ec4-4f4e970d05c7 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:53.177422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce22c-52da-41f3-8ec4-4f4e970d05c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:53.178001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce22c-52da-41f3-8ec4-4f4e970d05c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:53.184354 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8dcce-efeb-49fc-93bb-8f8da44d4072 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:53.187452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-493ce22c-52da-41f3-8ec4-4f4e970d05c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:53.216422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 842/1683] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:53.217890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77eb1cd-2975-4e8c-890f-e46f928eb80f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:53.218210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77eb1cd-2975-4e8c-890f-e46f928eb80f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:53.227619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77eb1cd-2975-4e8c-890f-e46f928eb80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:53.273304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e77eb1cd-2975-4e8c-890f-e46f928eb80f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:53.277628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e77eb1cd-2975-4e8c-890f-e46f928eb80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:53.378817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ce22c-52da-41f3-8ec4-4f4e970d05c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:53.380608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ce22c-52da-41f3-8ec4-4f4e970d05c7 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:53.399393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 842/1684] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:53.402037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f2b1c-2754-45bf-a536-f7af12de60b5 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:53.402586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f2b1c-2754-45bf-a536-f7af12de60b5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:53.405193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f2b1c-2754-45bf-a536-f7af12de60b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:53.450271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 843/1685] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:53.454677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bea39-5591-4115-bc06-248c82320ec5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:53.455921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bea39-5591-4115-bc06-248c82320ec5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:53.456240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bea39-5591-4115-bc06-248c82320ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:53.498391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-cc6f2b1c-2754-45bf-a536-f7af12de60b5 None None] MFA Rules not processed for user `0267350339c44000bcc49906cfa8f0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:53.503997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-cc6f2b1c-2754-45bf-a536-f7af12de60b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:53.537651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-4e8bea39-5591-4115-bc06-248c82320ec5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:53.538907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4e8bea39-5591-4115-bc06-248c82320ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:53.571634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 844/1686] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:53.572793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bea39-5591-4115-bc06-248c82320ec5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:53.594724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bea39-5591-4115-bc06-248c82320ec5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:53.606386 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:53.606531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a598891-3c80-439b-b725-50d6a3a49a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:53.607505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a598891-3c80-439b-b725-50d6a3a49a19 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:53.607505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a598891-3c80-439b-b725-50d6a3a49a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:53.607791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a598891-3c80-439b-b725-50d6a3a49a19 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:53.608536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a598891-3c80-439b-b725-50d6a3a49a19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:53.608857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a598891-3c80-439b-b725-50d6a3a49a19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:53.610544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a598891-3c80-439b-b725-50d6a3a49a19 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:53.646463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 845/1687] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:53.648143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a598891-3c80-439b-b725-50d6a3a49a19 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:53.673952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a598891-3c80-439b-b725-50d6a3a49a19 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:53.687233 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318cfdd-8410-4435-aade-0f7ee77b1fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:53.687921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318cfdd-8410-4435-aade-0f7ee77b1fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:53.688206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318cfdd-8410-4435-aade-0f7ee77b1fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:53.688483 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318cfdd-8410-4435-aade-0f7ee77b1fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:53.689183 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318cfdd-8410-4435-aade-0f7ee77b1fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:53.691144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318cfdd-8410-4435-aade-0f7ee77b1fe7 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:53.706836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 843/1688] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:53.726773 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21489b6e-5097-4faf-9817-3efb2ea66f2b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:53.727016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21489b6e-5097-4faf-9817-3efb2ea66f2b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:53.727297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21489b6e-5097-4faf-9817-3efb2ea66f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:53.759309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 846/1689] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:53 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:53.760547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318cfdd-8410-4435-aade-0f7ee77b1fe7 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:53.771283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-21489b6e-5097-4faf-9817-3efb2ea66f2b None None] MFA Rules not processed for user `0267350339c44000bcc49906cfa8f0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:53.772642 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-21489b6e-5097-4faf-9817-3efb2ea66f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:53.936895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 844/1690] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:53.950669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21489b6e-5097-4faf-9817-3efb2ea66f2b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:53.952755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-21489b6e-5097-4faf-9817-3efb2ea66f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:53.970802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318cfdd-8410-4435-aade-0f7ee77b1fe7 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:53.983981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4babcaf-7986-48bb-af73-58f844b1f36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:53.985137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4babcaf-7986-48bb-af73-58f844b1f36d admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:53.985685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4babcaf-7986-48bb-af73-58f844b1f36d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:53.986230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4babcaf-7986-48bb-af73-58f844b1f36d admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:53.987735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4babcaf-7986-48bb-af73-58f844b1f36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:53.990555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4babcaf-7986-48bb-af73-58f844b1f36d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:54.043521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 847/1691] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:53 2022] POST /identity/v3/users => generated 292 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:54.047593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4babcaf-7986-48bb-af73-58f844b1f36d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:54.095499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4babcaf-7986-48bb-af73-58f844b1f36d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:54.109715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36dff-085b-411c-9c1f-37b3128821ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:54.111273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36dff-085b-411c-9c1f-37b3128821ca admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:54.113754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36dff-085b-411c-9c1f-37b3128821ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:54.115094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36dff-085b-411c-9c1f-37b3128821ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:54.115879 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da36dff-085b-411c-9c1f-37b3128821ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:54.120776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da36dff-085b-411c-9c1f-37b3128821ca admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:54.173480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 848/1692] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:54.176688 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36dff-085b-411c-9c1f-37b3128821ca admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:54.196478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21489b6e-5097-4faf-9817-3efb2ea66f2b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:54.209693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36dff-085b-411c-9c1f-37b3128821ca admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:54.226977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed8efd-0c5a-4762-bb83-ba7cdfdf6dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:54.229840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed8efd-0c5a-4762-bb83-ba7cdfdf6dd3 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:54.229840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed8efd-0c5a-4762-bb83-ba7cdfdf6dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:54.229840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed8efd-0c5a-4762-bb83-ba7cdfdf6dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:54.230672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44c498-91b0-42fc-be13-73747cc61b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:54.232652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44c498-91b0-42fc-be13-73747cc61b75 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:54.232997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44c498-91b0-42fc-be13-73747cc61b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:54.233223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44c498-91b0-42fc-be13-73747cc61b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:54.234079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44c498-91b0-42fc-be13-73747cc61b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:54.246282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44c498-91b0-42fc-be13-73747cc61b75 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:54.265567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed8efd-0c5a-4762-bb83-ba7cdfdf6dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:54.279180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 849/1693] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:54.280333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed8efd-0c5a-4762-bb83-ba7cdfdf6dd3 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:54.281503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44c498-91b0-42fc-be13-73747cc61b75 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:54.306570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44c498-91b0-42fc-be13-73747cc61b75 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:54.319680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 845/1694] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:52:54.330431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf462b-0443-43b5-80dd-0c9ac76e34bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:54.331623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf462b-0443-43b5-80dd-0c9ac76e34bf admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:54.332088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf462b-0443-43b5-80dd-0c9ac76e34bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:54.332657 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf462b-0443-43b5-80dd-0c9ac76e34bf admin admin] PATH_INFO: `/v3/projects/fbb386cee3c940f786454cf18c347b38/users/c36ac1426cb84b519bc7553056bf57d6/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:54.377896 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf462b-0443-43b5-80dd-0c9ac76e34bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb386cee3c940f786454cf18c347b38, user_id=c36ac1426cb84b519bc7553056bf57d6, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:54.380740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf462b-0443-43b5-80dd-0c9ac76e34bf admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:54.383895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:54.412446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 850/1695] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:54 2022] PUT /identity/v3/projects/fbb386cee3c940f786454cf18c347b38/users/c36ac1426cb84b519bc7553056bf57d6/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:54.414836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317fc8c-28e3-4cc8-92f9-0907e7be627d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:54.415453 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317fc8c-28e3-4cc8-92f9-0907e7be627d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:54.415974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317fc8c-28e3-4cc8-92f9-0907e7be627d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:54.524297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf462b-0443-43b5-80dd-0c9ac76e34bf admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:54.527239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-bfbf462b-0443-43b5-80dd-0c9ac76e34bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:54.532773 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5317fc8c-28e3-4cc8-92f9-0907e7be627d None None] MFA Rules not processed for user `c36ac1426cb84b519bc7553056bf57d6`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:54.534547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5317fc8c-28e3-4cc8-92f9-0907e7be627d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:54.814291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf462b-0443-43b5-80dd-0c9ac76e34bf admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:54.823962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:54.824900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:54.825181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:54.825455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] PATH_INFO: `/v3/projects/4de3855f7cab4f1b828d5ab385e4f74e` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:54.826901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de3855f7cab4f1b828d5ab385e4f74e)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:54.829328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:54.857335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:54.888283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 846/1696] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:54.893558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317fc8c-28e3-4cc8-92f9-0907e7be627d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:54.921720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317fc8c-28e3-4cc8-92f9-0907e7be627d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:54.945367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a53874-958b-4bc5-9148-6851d259c4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:54.946090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a53874-958b-4bc5-9148-6851d259c4ee admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:54.947224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a53874-958b-4bc5-9148-6851d259c4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:54.948957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a53874-958b-4bc5-9148-6851d259c4ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:54.950998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a53874-958b-4bc5-9148-6851d259c4ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:54.956138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a53874-958b-4bc5-9148-6851d259c4ee admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:55.019940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de3855f7cab4f1b828d5ab385e4f74e', 'request_id': 'req-20dc0425-842f-44d6-bf00-6402620745f0'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:55.028932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 847/1697] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:54 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:55.030569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a53874-958b-4bc5-9148-6851d259c4ee admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:55.049045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de3855f7cab4f1b828d5ab385e4f74e', 'request_id': 'req-20dc0425-842f-44d6-bf00-6402620745f0'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:55.090912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de3855f7cab4f1b828d5ab385e4f74e', 'request_id': 'req-20dc0425-842f-44d6-bf00-6402620745f0'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:55.148562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] The token cache is being invalidate because project 4de3855f7cab4f1b828d5ab385e4f74e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:52:55.149250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:55.150163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:55.153189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 851/1698] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:52:54 2022] DELETE /identity/v3/projects/4de3855f7cab4f1b828d5ab385e4f74e => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:52:55.157257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:55.159162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:55.277769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a53874-958b-4bc5-9148-6851d259c4ee admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:55.290714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc9427-1160-4d9d-b7b3-6db3a9bc3284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:55.291847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc9427-1160-4d9d-b7b3-6db3a9bc3284 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:55.292175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc9427-1160-4d9d-b7b3-6db3a9bc3284 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:55.292603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc9427-1160-4d9d-b7b3-6db3a9bc3284 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:55.301363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc9427-1160-4d9d-b7b3-6db3a9bc3284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:55.303764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc9427-1160-4d9d-b7b3-6db3a9bc3284 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:55.354309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc0425-842f-44d6-bf00-6402620745f0 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:55.374410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 848/1699] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:55 2022] POST /identity/v3/users => generated 300 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:55.374581 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:55.375793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc9427-1160-4d9d-b7b3-6db3a9bc3284 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:55.377847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:55.378244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:55.378529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] PATH_INFO: `/v3/users/2199bee011fe412dbf04a9f919504ea3` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:55.379337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-95dc9427-1160-4d9d-b7b3-6db3a9bc3284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:55.383584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2199bee011fe412dbf04a9f919504ea3)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:55.390437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:55.426443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc9427-1160-4d9d-b7b3-6db3a9bc3284 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:55.450253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cbb38-eb8a-4e0e-920a-8f11ca97ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:55.452413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cbb38-eb8a-4e0e-920a-8f11ca97ae20 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:55.460064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cbb38-eb8a-4e0e-920a-8f11ca97ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:55.462030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cbb38-eb8a-4e0e-920a-8f11ca97ae20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:55.463075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cbb38-eb8a-4e0e-920a-8f11ca97ae20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:55.473465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cbb38-eb8a-4e0e-920a-8f11ca97ae20 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:55.513628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:55.515157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 849/1700] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:55.526122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cbb38-eb8a-4e0e-920a-8f11ca97ae20 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:55.559022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cbb38-eb8a-4e0e-920a-8f11ca97ae20 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:55.575883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c6a0e-73ff-4259-b449-6bf3d41cfc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:55.577379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c6a0e-73ff-4259-b449-6bf3d41cfc9a admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:55.577904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c6a0e-73ff-4259-b449-6bf3d41cfc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:55.578856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c6a0e-73ff-4259-b449-6bf3d41cfc9a admin admin] PATH_INFO: `/v3/projects/90956bebf5a44cb4a214106996e50db3/users/69ff4b6469a147c1ba753d2b71fbce5c/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:55.578985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2199bee011fe412dbf04a9f919504ea3'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:55.595815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2199bee011fe412dbf04a9f919504ea3'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:55.606731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2199bee011fe412dbf04a9f919504ea3'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:55.627289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c6a0e-73ff-4259-b449-6bf3d41cfc9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=90956bebf5a44cb4a214106996e50db3, user_id=69ff4b6469a147c1ba753d2b71fbce5c, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:55.636742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2199bee011fe412dbf04a9f919504ea3'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:55.638023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2199bee011fe412dbf04a9f919504ea3'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:55.643353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c6a0e-73ff-4259-b449-6bf3d41cfc9a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:55.663798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 852/1701] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:52:55 2022] DELETE /identity/v3/users/2199bee011fe412dbf04a9f919504ea3 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:55.704294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 850/1702] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:55 2022] PUT /identity/v3/projects/90956bebf5a44cb4a214106996e50db3/users/69ff4b6469a147c1ba753d2b71fbce5c/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:55.708757 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:55.711225 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:55.922020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45206e7-772b-46ab-88c3-3e6c41e9d31a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:56.096595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bca196-4c30-4bdf-8228-939823e8255a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:56.097446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bca196-4c30-4bdf-8228-939823e8255a admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:56.097760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bca196-4c30-4bdf-8228-939823e8255a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:56.099091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bca196-4c30-4bdf-8228-939823e8255a admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:56.102057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bca196-4c30-4bdf-8228-939823e8255a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:56.108140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bca196-4c30-4bdf-8228-939823e8255a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:56.165207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 853/1703] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:56.171587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c6a0e-73ff-4259-b449-6bf3d41cfc9a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:56.190643 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c6a0e-73ff-4259-b449-6bf3d41cfc9a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:56.208190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bfa1b-abe9-459b-ada5-043c2b1df7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:56.209112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bfa1b-abe9-459b-ada5-043c2b1df7a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:56.209724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bfa1b-abe9-459b-ada5-043c2b1df7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:56.209870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bfa1b-abe9-459b-ada5-043c2b1df7a1 admin admin] PATH_INFO: `/v3/projects/90956bebf5a44cb4a214106996e50db3/users/69ff4b6469a147c1ba753d2b71fbce5c/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:56.218524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bfa1b-abe9-459b-ada5-043c2b1df7a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90956bebf5a44cb4a214106996e50db3, user_id=69ff4b6469a147c1ba753d2b71fbce5c, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:56.221541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bfa1b-abe9-459b-ada5-043c2b1df7a1 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:56.266484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 851/1704] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:52:56 2022] PUT /identity/v3/projects/90956bebf5a44cb4a214106996e50db3/users/69ff4b6469a147c1ba753d2b71fbce5c/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:56.268409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97d559-8e7e-47e5-9419-876ed0e71f40 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:56.268780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97d559-8e7e-47e5-9419-876ed0e71f40 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:56.269294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97d559-8e7e-47e5-9419-876ed0e71f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:56.341261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-2d97d559-8e7e-47e5-9419-876ed0e71f40 None None] MFA Rules not processed for user `69ff4b6469a147c1ba753d2b71fbce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:56.342809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2d97d559-8e7e-47e5-9419-876ed0e71f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:56.474093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:56.522359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f27b39-de91-4c9c-8920-31f5aeedee29 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:56.522845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f27b39-de91-4c9c-8920-31f5aeedee29 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:56.522948 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f27b39-de91-4c9c-8920-31f5aeedee29 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:56.524272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 852/1705] 10.222.0.66 () {54 vars in 912 bytes} [Fri Jul 1 19:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:52:56.530836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e9ba-1b1a-492f-83ed-d4dbc6c10009 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:56.531322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e9ba-1b1a-492f-83ed-d4dbc6c10009 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:56.531736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77e9ba-1b1a-492f-83ed-d4dbc6c10009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:56.577307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 854/1706] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:56.598902 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97d559-8e7e-47e5-9419-876ed0e71f40 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:56.605916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2d97d559-8e7e-47e5-9419-876ed0e71f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:56.660900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6c77e9ba-1b1a-492f-83ed-d4dbc6c10009 None None] MFA Rules not processed for user `370f1c0f7af04a56bdf8fa452550380a`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:56.662763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6c77e9ba-1b1a-492f-83ed-d4dbc6c10009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:56.782759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97d559-8e7e-47e5-9419-876ed0e71f40 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:56.798319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559d2c8-a9d2-442a-8414-471c96f00694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:56.798994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559d2c8-a9d2-442a-8414-471c96f00694 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:56.799274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559d2c8-a9d2-442a-8414-471c96f00694 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:56.799549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559d2c8-a9d2-442a-8414-471c96f00694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:56.813395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559d2c8-a9d2-442a-8414-471c96f00694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:56.814885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559d2c8-a9d2-442a-8414-471c96f00694 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:56.827949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 855/1707] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:56.840302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559d2c8-a9d2-442a-8414-471c96f00694 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:56.843779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b559d2c8-a9d2-442a-8414-471c96f00694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:56.912487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 853/1708] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:52:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:52:56.915730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e9ba-1b1a-492f-83ed-d4dbc6c10009 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:56.923829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6c77e9ba-1b1a-492f-83ed-d4dbc6c10009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:56.964023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77e9ba-1b1a-492f-83ed-d4dbc6c10009 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:56.988741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f6f77-978c-4bcf-a8ca-43f7b5014cd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:56.989924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f6f77-978c-4bcf-a8ca-43f7b5014cd2 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:56.990455 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f6f77-978c-4bcf-a8ca-43f7b5014cd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:56.991160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f6f77-978c-4bcf-a8ca-43f7b5014cd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:57.019050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f6f77-978c-4bcf-a8ca-43f7b5014cd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:57.022245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f6f77-978c-4bcf-a8ca-43f7b5014cd2 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:57.027644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:57.061205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 854/1709] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:57.062444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f6f77-978c-4bcf-a8ca-43f7b5014cd2 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:57.126844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559d2c8-a9d2-442a-8414-471c96f00694 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:57.156034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:57.159668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:57.160168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:57.160623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] PATH_INFO: `/v3/projects/388c31e0cd074222aac4acbebcdfd507` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:57.165145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=388c31e0cd074222aac4acbebcdfd507)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:57.168726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:57.295859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f6f77-978c-4bcf-a8ca-43f7b5014cd2 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:57.325295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '388c31e0cd074222aac4acbebcdfd507', 'request_id': 'req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.346216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '388c31e0cd074222aac4acbebcdfd507', 'request_id': 'req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.354818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '388c31e0cd074222aac4acbebcdfd507', 'request_id': 'req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.376924 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] The token cache is being invalidate because project 388c31e0cd074222aac4acbebcdfd507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:52:57.378579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.379626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b9a78ee3-d6c9-4e6b-8804-31fd58cbb4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.386334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 856/1710] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:52:56 2022] DELETE /identity/v3/projects/388c31e0cd074222aac4acbebcdfd507 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:57.393902 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200860d-a86f-477a-b7fa-809c4025e6c7 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:57.397157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200860d-a86f-477a-b7fa-809c4025e6c7 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:57.405330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200860d-a86f-477a-b7fa-809c4025e6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:57.418537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:57.446313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c200860d-a86f-477a-b7fa-809c4025e6c7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:57.450083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c200860d-a86f-477a-b7fa-809c4025e6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:57.464236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:57.465226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:57.465509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:57.465811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] PATH_INFO: `/v3/users/c36ac1426cb84b519bc7553056bf57d6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:57.467193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c36ac1426cb84b519bc7553056bf57d6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:57.469115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:57.584750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 857/1711] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:57.585778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c36ac1426cb84b519bc7553056bf57d6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.586151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200860d-a86f-477a-b7fa-809c4025e6c7 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:57.587280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c200860d-a86f-477a-b7fa-809c4025e6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:57.595400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c36ac1426cb84b519bc7553056bf57d6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.596123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c36ac1426cb84b519bc7553056bf57d6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.602850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200860d-a86f-477a-b7fa-809c4025e6c7 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:57.610368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:57.611360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:57.611775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:57.612145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] PATH_INFO: `/v3/users/3f1f6d0d8a6d4c649b0f4f58de0d59d1` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:57.613653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f1f6d0d8a6d4c649b0f4f58de0d59d1)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:57.615686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:57.615870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c36ac1426cb84b519bc7553056bf57d6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.616627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b50234b5-1706-493e-8870-b092d053c597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c36ac1426cb84b519bc7553056bf57d6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.633713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 855/1712] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:52:57 2022] DELETE /identity/v3/users/c36ac1426cb84b519bc7553056bf57d6 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:57.639901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c585c0-35f2-425a-a67d-acec1d7e6b9d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:57.640192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c585c0-35f2-425a-a67d-acec1d7e6b9d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:57.640609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c585c0-35f2-425a-a67d-acec1d7e6b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:57.673268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-24c585c0-35f2-425a-a67d-acec1d7e6b9d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:57.675200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-24c585c0-35f2-425a-a67d-acec1d7e6b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:57.699144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f1f6d0d8a6d4c649b0f4f58de0d59d1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.717738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:57.727789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f1f6d0d8a6d4c649b0f4f58de0d59d1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.751166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f1f6d0d8a6d4c649b0f4f58de0d59d1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.762715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f1f6d0d8a6d4c649b0f4f58de0d59d1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.764371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0b983b9f-c1ba-4163-8885-117e8650449f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f1f6d0d8a6d4c649b0f4f58de0d59d1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:57.780343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 858/1713] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:52:57 2022] DELETE /identity/v3/users/3f1f6d0d8a6d4c649b0f4f58de0d59d1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:57.781762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e61fd-2bc1-45fa-a8ae-046a267b9ed8 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:57.783451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e61fd-2bc1-45fa-a8ae-046a267b9ed8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:57.783894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e61fd-2bc1-45fa-a8ae-046a267b9ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:57.794887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 856/1714] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:57.796546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b030bf-e4b6-4616-b42b-f965a60872c5 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:57.796981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b030bf-e4b6-4616-b42b-f965a60872c5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:57.797261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b030bf-e4b6-4616-b42b-f965a60872c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:57.820564 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a7b030bf-e4b6-4616-b42b-f965a60872c5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:57.822148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a7b030bf-e4b6-4616-b42b-f965a60872c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:57.824468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-fe1e61fd-2bc1-45fa-a8ae-046a267b9ed8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:57.835684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fe1e61fd-2bc1-45fa-a8ae-046a267b9ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:57.910937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 857/1715] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:57.912190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b030bf-e4b6-4616-b42b-f965a60872c5 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:57.936107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b030bf-e4b6-4616-b42b-f965a60872c5 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:57.945199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6693c-dab5-4a00-9c7b-1be3c06c7f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:57.945964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6693c-dab5-4a00-9c7b-1be3c06c7f12 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:57.946245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6693c-dab5-4a00-9c7b-1be3c06c7f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:57.946519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6693c-dab5-4a00-9c7b-1be3c06c7f12 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:57.954054 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6693c-dab5-4a00-9c7b-1be3c06c7f12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:52:57.956317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6693c-dab5-4a00-9c7b-1be3c06c7f12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:57.958768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6693c-dab5-4a00-9c7b-1be3c06c7f12 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:57.986045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 858/1716] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:52:57.990886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303c253-dc08-4b8a-aefa-d11987c2a3b3 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:57.991367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303c253-dc08-4b8a-aefa-d11987c2a3b3 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:57.992294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303c253-dc08-4b8a-aefa-d11987c2a3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:58.021727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 859/1717] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:58.022623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e61fd-2bc1-45fa-a8ae-046a267b9ed8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:58.026805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fe1e61fd-2bc1-45fa-a8ae-046a267b9ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:58.079722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:58.105112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c303c253-dc08-4b8a-aefa-d11987c2a3b3 None None] MFA Rules not processed for user `3198a63f6d1a4cdd88d76189658e79c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:58.106407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c303c253-dc08-4b8a-aefa-d11987c2a3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:58.215682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e61fd-2bc1-45fa-a8ae-046a267b9ed8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:58.227790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7f8ec-2c6c-4f8d-bbca-d376e9a7a40d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:58.229629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7f8ec-2c6c-4f8d-bbca-d376e9a7a40d service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:58.229967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7f8ec-2c6c-4f8d-bbca-d376e9a7a40d service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:58.230271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7f8ec-2c6c-4f8d-bbca-d376e9a7a40d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:58.261885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7f8ec-2c6c-4f8d-bbca-d376e9a7a40d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:58.263595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7f8ec-2c6c-4f8d-bbca-d376e9a7a40d service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:58.301288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 859/1718] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:58.302264 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 860/1719] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:58.303339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7f8ec-2c6c-4f8d-bbca-d376e9a7a40d service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:58.304832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303c253-dc08-4b8a-aefa-d11987c2a3b3 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:58.323577 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7f8ec-2c6c-4f8d-bbca-d376e9a7a40d service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:58.332771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add33bf1-2e3e-4c87-8684-279928e706d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:58.333459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add33bf1-2e3e-4c87-8684-279928e706d8 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:58.333778 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add33bf1-2e3e-4c87-8684-279928e706d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:58.335225 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add33bf1-2e3e-4c87-8684-279928e706d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:58.359499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303c253-dc08-4b8a-aefa-d11987c2a3b3 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:58.372792 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1052b60-1936-4cdf-a3ca-f03db822ee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:58.374132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1052b60-1936-4cdf-a3ca-f03db822ee76 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:58.374410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1052b60-1936-4cdf-a3ca-f03db822ee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:58.374678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1052b60-1936-4cdf-a3ca-f03db822ee76 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:58.376064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add33bf1-2e3e-4c87-8684-279928e706d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:58.380704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1052b60-1936-4cdf-a3ca-f03db822ee76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:58.382663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1052b60-1936-4cdf-a3ca-f03db822ee76 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:58.383716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add33bf1-2e3e-4c87-8684-279928e706d8 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:58.415359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 861/1720] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:58.417880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb1bc1-f347-4c48-b568-5d20172395f8 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:58.418167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb1bc1-f347-4c48-b568-5d20172395f8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:58.418467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb1bc1-f347-4c48-b568-5d20172395f8 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:58.420155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 862/1721] 10.222.0.66 () {54 vars in 960 bytes} [Fri Jul 1 19:52:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:52:58.431214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f4a9-c6e3-4466-aea7-d51832da91ac None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:58.431506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f4a9-c6e3-4466-aea7-d51832da91ac None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:58.431975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f4a9-c6e3-4466-aea7-d51832da91ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:58.449664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 860/1722] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:52:58 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:58.450947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1052b60-1936-4cdf-a3ca-f03db822ee76 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:58.453299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b1052b60-1936-4cdf-a3ca-f03db822ee76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:58.484727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f7d8f4a9-c6e3-4466-aea7-d51832da91ac None None] MFA Rules not processed for user `0267350339c44000bcc49906cfa8f0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:58.486287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f7d8f4a9-c6e3-4466-aea7-d51832da91ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:58.532190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:58.642564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 863/1723] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:58.657597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb04696-2679-4598-8e79-0dedf1c2629e None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:58.660009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb04696-2679-4598-8e79-0dedf1c2629e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:58.663878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb04696-2679-4598-8e79-0dedf1c2629e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:58.734605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-2fb04696-2679-4598-8e79-0dedf1c2629e None None] MFA Rules not processed for user `bba7d9205e694f2182d6091061d1219c`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:58.737773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2fb04696-2679-4598-8e79-0dedf1c2629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:58.754611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1052b60-1936-4cdf-a3ca-f03db822ee76 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:58.784364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e2a33-a49b-42e2-bb3d-589d6060f21e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:58.785274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e2a33-a49b-42e2-bb3d-589d6060f21e service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:58.785756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e2a33-a49b-42e2-bb3d-589d6060f21e service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:58.786202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e2a33-a49b-42e2-bb3d-589d6060f21e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:58.878198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e2a33-a49b-42e2-bb3d-589d6060f21e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:58.880289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e2a33-a49b-42e2-bb3d-589d6060f21e service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:58.897686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 861/1724] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:58.898948 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e2a33-a49b-42e2-bb3d-589d6060f21e service nova] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:58.926158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 864/1725] 10.222.0.66 () {58 vars in 1057 bytes} [Fri Jul 1 19:52:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:58.927129 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb04696-2679-4598-8e79-0dedf1c2629e None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:58.935775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2fb04696-2679-4598-8e79-0dedf1c2629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:58.995126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e2a33-a49b-42e2-bb3d-589d6060f21e service nova] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:59.003223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d97c6-b983-41b2-b9b9-d9494890c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:59.004063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d97c6-b983-41b2-b9b9-d9494890c6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:59.004346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d97c6-b983-41b2-b9b9-d9494890c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:59.004650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d97c6-b983-41b2-b9b9-d9494890c6d1 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:59.005854 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d97c6-b983-41b2-b9b9-d9494890c6d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:59.009755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d97c6-b983-41b2-b9b9-d9494890c6d1 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:59.055528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 862/1726] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:52:58 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:52:59.057647 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b705fe-87b4-43ee-a410-bacdd6d4308c None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:59.058203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b705fe-87b4-43ee-a410-bacdd6d4308c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:59.058203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b705fe-87b4-43ee-a410-bacdd6d4308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:59.080009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:59.102757 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb04696-2679-4598-8e79-0dedf1c2629e None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:59.108935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-86b705fe-87b4-43ee-a410-bacdd6d4308c None None] MFA Rules not processed for user `71647f9f93ec4235818d27640058d25c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:59.111094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:59.111227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-86b705fe-87b4-43ee-a410-bacdd6d4308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:59.112079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:59.112424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:59.112698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] PATH_INFO: `/v3/users/0b16b4ddea3a43489100e90281741160` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:59.114266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b16b4ddea3a43489100e90281741160)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:59.116208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:59.205947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b16b4ddea3a43489100e90281741160'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:59.217536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:59.232921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b16b4ddea3a43489100e90281741160'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:59.274054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b16b4ddea3a43489100e90281741160'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:59.290887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b16b4ddea3a43489100e90281741160'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:59.292182 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b16b4ddea3a43489100e90281741160'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:52:59.304472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 863/1728] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:59.305502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b705fe-87b4-43ee-a410-bacdd6d4308c None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:59.305628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 865/1728] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:52:58 2022] DELETE /identity/v3/users/0b16b4ddea3a43489100e90281741160 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:52:59.311238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:59.314447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:59.335578 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b705fe-87b4-43ee-a410-bacdd6d4308c None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:59.543485 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5414e7f-86b3-41f5-b67f-8f93a57606db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:59.544461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5414e7f-86b3-41f5-b67f-8f93a57606db service placement] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:59.544919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5414e7f-86b3-41f5-b67f-8f93a57606db service placement] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:59.545357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5414e7f-86b3-41f5-b67f-8f93a57606db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:59.548801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d5414e7f-86b3-41f5-b67f-8f93a57606db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:59.555866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0674f0-588a-4b90-a29b-e1b7242ef93a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:59.567576 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989de1e8-3a68-47b1-9384-9de9e69cb78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:59.568531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989de1e8-3a68-47b1-9384-9de9e69cb78b admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:59.568531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989de1e8-3a68-47b1-9384-9de9e69cb78b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:59.568806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989de1e8-3a68-47b1-9384-9de9e69cb78b admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:59.576510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5414e7f-86b3-41f5-b67f-8f93a57606db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:59.577491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989de1e8-3a68-47b1-9384-9de9e69cb78b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:59.578798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5414e7f-86b3-41f5-b67f-8f93a57606db service placement] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:59.579255 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989de1e8-3a68-47b1-9384-9de9e69cb78b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:59.597087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 866/1729] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:59.604227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ce776-5390-46a6-868e-1e6df397c519 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:59.604477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ce776-5390-46a6-868e-1e6df397c519 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:59.604697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ce776-5390-46a6-868e-1e6df397c519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:59.607885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 864/1730] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:52:59.613331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157d8e3-6660-4a9c-93a0-02d652d415b6 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:59.613903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157d8e3-6660-4a9c-93a0-02d652d415b6 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:59.614351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157d8e3-6660-4a9c-93a0-02d652d415b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:59.662634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c157d8e3-6660-4a9c-93a0-02d652d415b6 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:59.664469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c157d8e3-6660-4a9c-93a0-02d652d415b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:59.670947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e26ce776-5390-46a6-868e-1e6df397c519 None None] MFA Rules not processed for user `71647f9f93ec4235818d27640058d25c`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:59.674484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e26ce776-5390-46a6-868e-1e6df397c519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:59.688781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 865/1731] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:59.688781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bd4bb-48d0-4524-bdd3-b78be9aa62b4 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:59.692658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bd4bb-48d0-4524-bdd3-b78be9aa62b4 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:59.703686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bd4bb-48d0-4524-bdd3-b78be9aa62b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:59.763199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-497bd4bb-48d0-4524-bdd3-b78be9aa62b4 None None] MFA Rules not processed for user `3307be3159654f55be46e75c82584bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:52:59.766780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-497bd4bb-48d0-4524-bdd3-b78be9aa62b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:59.840175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 867/1732] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:59.840854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ce776-5390-46a6-868e-1e6df397c519 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:59.852606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ce776-5390-46a6-868e-1e6df397c519 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:59.858042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad280c-d51a-405a-a91d-9d6387100604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:59.859353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad280c-d51a-405a-a91d-9d6387100604 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:59.859647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad280c-d51a-405a-a91d-9d6387100604 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:59.859925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad280c-d51a-405a-a91d-9d6387100604 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:59.860672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad280c-d51a-405a-a91d-9d6387100604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:59.862340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad280c-d51a-405a-a91d-9d6387100604 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:59.865303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:52:59.901306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 868/1733] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:52:59.901306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad280c-d51a-405a-a91d-9d6387100604 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:52:59.901972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-28ad280c-d51a-405a-a91d-9d6387100604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:52:59.925081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad280c-d51a-405a-a91d-9d6387100604 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:52:59.936843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:52:59.941086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:52:59.941424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:52:59.941741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] PATH_INFO: `/v3/projects/fcb6931bdfc84ca090b172a0fcf4f6ae` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:52:59.946782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb6931bdfc84ca090b172a0fcf4f6ae)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:52:59.950178 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:52:59.994436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 866/1734] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:52:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:52:59.997518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bd4bb-48d0-4524-bdd3-b78be9aa62b4 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:00.005511 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-497bd4bb-48d0-4524-bdd3-b78be9aa62b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:00.069680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb6931bdfc84ca090b172a0fcf4f6ae', 'request_id': 'req-322aa17d-24d5-47da-b90b-e69a03ec0cec'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:00.096647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb6931bdfc84ca090b172a0fcf4f6ae', 'request_id': 'req-322aa17d-24d5-47da-b90b-e69a03ec0cec'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:00.108015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb6931bdfc84ca090b172a0fcf4f6ae', 'request_id': 'req-322aa17d-24d5-47da-b90b-e69a03ec0cec'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:00.148257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] The token cache is being invalidate because project fcb6931bdfc84ca090b172a0fcf4f6ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:00.148562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:00.149270 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:00.162684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 869/1735] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:52:59 2022] DELETE /identity/v3/projects/fcb6931bdfc84ca090b172a0fcf4f6ae => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:00.164274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:00.165800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:00.174744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:00.190938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bd4bb-48d0-4524-bdd3-b78be9aa62b4 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:00.312145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aa17d-24d5-47da-b90b-e69a03ec0cec admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:00.319767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c431b-60ed-44ec-8aaa-3c4dbcba276f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:00.322582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c431b-60ed-44ec-8aaa-3c4dbcba276f service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:00.322867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c431b-60ed-44ec-8aaa-3c4dbcba276f service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:00.323336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c431b-60ed-44ec-8aaa-3c4dbcba276f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:00.327218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5c0c431b-60ed-44ec-8aaa-3c4dbcba276f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:00.357353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c99175-006e-4bca-badf-0a83a3aa132d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:00.359211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c99175-006e-4bca-badf-0a83a3aa132d service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:00.359495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c99175-006e-4bca-badf-0a83a3aa132d service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:00.359765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c99175-006e-4bca-badf-0a83a3aa132d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:00.365013 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-42c99175-006e-4bca-badf-0a83a3aa132d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:00.479390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c431b-60ed-44ec-8aaa-3c4dbcba276f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:00.482526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c431b-60ed-44ec-8aaa-3c4dbcba276f service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:00.507203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c99175-006e-4bca-badf-0a83a3aa132d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:00.510312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c99175-006e-4bca-badf-0a83a3aa132d service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:00.511273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 870/1736] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:00.512157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c431b-60ed-44ec-8aaa-3c4dbcba276f service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:00.514905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5c0c431b-60ed-44ec-8aaa-3c4dbcba276f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:00.535073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 867/1737] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:52:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:00.536445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38d2e3-a17e-4e5c-8e9a-0e85f7b1a507 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:00.537196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38d2e3-a17e-4e5c-8e9a-0e85f7b1a507 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:00.537196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38d2e3-a17e-4e5c-8e9a-0e85f7b1a507 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:00.543046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 868/1738] 10.222.0.66 () {54 vars in 912 bytes} [Fri Jul 1 19:53:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:53:00.544324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c431b-60ed-44ec-8aaa-3c4dbcba276f service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:00.544938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38d2e3-a17e-4e5c-8e9a-0e85f7b1a507 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:00.549266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9e38d2e3-a17e-4e5c-8e9a-0e85f7b1a507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:00.556152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db29a21-f638-4eea-aa94-c9b4ae4bd2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:00.558453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db29a21-f638-4eea-aa94-c9b4ae4bd2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:00.558706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db29a21-f638-4eea-aa94-c9b4ae4bd2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:00.559815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db29a21-f638-4eea-aa94-c9b4ae4bd2dd admin admin] PATH_INFO: `/v3/projects/ba1ea012747e4a36badacb6d401c3735/users/4d12ea89fdfc454293a3c2b5e03e2dc3/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:00.589122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38d2e3-a17e-4e5c-8e9a-0e85f7b1a507 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:00.596501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db29a21-f638-4eea-aa94-c9b4ae4bd2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba1ea012747e4a36badacb6d401c3735, user_id=4d12ea89fdfc454293a3c2b5e03e2dc3, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:00.601533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:00.602449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:00.605099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db29a21-f638-4eea-aa94-c9b4ae4bd2dd admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:00.605822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:00.606186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] PATH_INFO: `/v3/users/b2eb8c198041421f8459daf6c357e9e8` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:00.609091 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2eb8c198041421f8459daf6c357e9e8)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:00.624825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:00.669744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 871/1739] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:00 2022] PUT /identity/v3/projects/ba1ea012747e4a36badacb6d401c3735/users/4d12ea89fdfc454293a3c2b5e03e2dc3/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:00.671954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8486a-f406-4f5c-abf2-16a93f81f917 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:00.672460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8486a-f406-4f5c-abf2-16a93f81f917 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:00.680496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8486a-f406-4f5c-abf2-16a93f81f917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:00.703975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:00.727118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e3f8486a-f406-4f5c-abf2-16a93f81f917 None None] MFA Rules not processed for user `370f1c0f7af04a56bdf8fa452550380a`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:00.731173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e3f8486a-f406-4f5c-abf2-16a93f81f917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:00.768556 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2eb8c198041421f8459daf6c357e9e8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:00.797478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2eb8c198041421f8459daf6c357e9e8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:00.798270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2eb8c198041421f8459daf6c357e9e8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:00.832633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2eb8c198041421f8459daf6c357e9e8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:00.833507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ac23272c-0cea-4342-957f-096f0eebd074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2eb8c198041421f8459daf6c357e9e8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:00.853149 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 869/1740] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:00 2022] DELETE /identity/v3/users/b2eb8c198041421f8459daf6c357e9e8 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:00.855939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6db443-b300-4f4a-9396-b2de2c192fd0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:00.856089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6db443-b300-4f4a-9396-b2de2c192fd0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:00.856681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6db443-b300-4f4a-9396-b2de2c192fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:00.886416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:00.898627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5c6db443-b300-4f4a-9396-b2de2c192fd0 None None] MFA Rules not processed for user `4d12ea89fdfc454293a3c2b5e03e2dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:00.900035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5c6db443-b300-4f4a-9396-b2de2c192fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:00.916515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 872/1741] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:53:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:00.923421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8486a-f406-4f5c-abf2-16a93f81f917 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:00.928534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e3f8486a-f406-4f5c-abf2-16a93f81f917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:01.055815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 870/1742] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:01.057986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6db443-b300-4f4a-9396-b2de2c192fd0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:01.059550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5c6db443-b300-4f4a-9396-b2de2c192fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:01.135645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8486a-f406-4f5c-abf2-16a93f81f917 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:01.305731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6db443-b300-4f4a-9396-b2de2c192fd0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:01.313556 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8d0c3-5bbb-46dc-ac39-f6dee6d8801e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:01.314316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8d0c3-5bbb-46dc-ac39-f6dee6d8801e admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:01.314592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8d0c3-5bbb-46dc-ac39-f6dee6d8801e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:01.314857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8d0c3-5bbb-46dc-ac39-f6dee6d8801e admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:01.315630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8d0c3-5bbb-46dc-ac39-f6dee6d8801e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:01.317491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8d0c3-5bbb-46dc-ac39-f6dee6d8801e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:01.376679 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 871/1743] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:01 2022] POST /identity/v3/projects => generated 387 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:01.380872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8d0c3-5bbb-46dc-ac39-f6dee6d8801e admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:01.381937 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:01.384899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:01.386714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-bcf8d0c3-5bbb-46dc-ac39-f6dee6d8801e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:01.389030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:01.389983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] PATH_INFO: `/v3/projects/fbb386cee3c940f786454cf18c347b38` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:01.396388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb386cee3c940f786454cf18c347b38)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:01.404164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:01.570082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb386cee3c940f786454cf18c347b38', 'request_id': 'req-5b603f8b-1879-408f-b580-cbfea18bc581'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:01.582019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8d0c3-5bbb-46dc-ac39-f6dee6d8801e admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:01.602722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb386cee3c940f786454cf18c347b38', 'request_id': 'req-5b603f8b-1879-408f-b580-cbfea18bc581'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:01.616024 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb386cee3c940f786454cf18c347b38', 'request_id': 'req-5b603f8b-1879-408f-b580-cbfea18bc581'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:01.618201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:01.645455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] The token cache is being invalidate because project fbb386cee3c940f786454cf18c347b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:01.646004 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:01.647017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:01.649159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 873/1744] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:00 2022] DELETE /identity/v3/projects/fbb386cee3c940f786454cf18c347b38 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:01.650080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:01.655343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:01.670770 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:01.694973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0de2f-5390-48d3-b4e4-c44d0d85debd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:01.695990 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0de2f-5390-48d3-b4e4-c44d0d85debd service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:01.696469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0de2f-5390-48d3-b4e4-c44d0d85debd service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:01.697461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0de2f-5390-48d3-b4e4-c44d0d85debd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:01.700406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-69f0de2f-5390-48d3-b4e4-c44d0d85debd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:01.735503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0de2f-5390-48d3-b4e4-c44d0d85debd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:01.740636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0de2f-5390-48d3-b4e4-c44d0d85debd service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:01.769800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 872/1745] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:01.770695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0de2f-5390-48d3-b4e4-c44d0d85debd service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:01.772588 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b603f8b-1879-408f-b580-cbfea18bc581 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:01.776870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-69f0de2f-5390-48d3-b4e4-c44d0d85debd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:01.781386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b7e14-00ae-4fcb-bce3-ca4b8a084a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:01.782847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b7e14-00ae-4fcb-bce3-ca4b8a084a40 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:01.783381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b7e14-00ae-4fcb-bce3-ca4b8a084a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:01.783850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b7e14-00ae-4fcb-bce3-ca4b8a084a40 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:01.786076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b7e14-00ae-4fcb-bce3-ca4b8a084a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:01.789492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b7e14-00ae-4fcb-bce3-ca4b8a084a40 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:01.822770 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0de2f-5390-48d3-b4e4-c44d0d85debd service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:01.826681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 874/1746] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:01 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:01.830338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:01.831006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b7e14-00ae-4fcb-bce3-ca4b8a084a40 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:01.831758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:01.832244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:01.832720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] PATH_INFO: `/v3/users/69ff4b6469a147c1ba753d2b71fbce5c` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:01.835245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ff4b6469a147c1ba753d2b71fbce5c)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:01.837428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:01.864025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b7e14-00ae-4fcb-bce3-ca4b8a084a40 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:01.878940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35329e1-d25f-4231-b320-4118bd927cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:01.883133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35329e1-d25f-4231-b320-4118bd927cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:01.883427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35329e1-d25f-4231-b320-4118bd927cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:01.883711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35329e1-d25f-4231-b320-4118bd927cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:01.886482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35329e1-d25f-4231-b320-4118bd927cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:01.892468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35329e1-d25f-4231-b320-4118bd927cd1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:01.940045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 875/1747] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:01.942649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35329e1-d25f-4231-b320-4118bd927cd1 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:01.984625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35329e1-d25f-4231-b320-4118bd927cd1 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:01.998400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ff4b6469a147c1ba753d2b71fbce5c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.013652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d472fb-3c3a-4e00-9982-a0c611573f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:02.015009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d472fb-3c3a-4e00-9982-a0c611573f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:02.015466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d472fb-3c3a-4e00-9982-a0c611573f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:02.015900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d472fb-3c3a-4e00-9982-a0c611573f3b admin admin] PATH_INFO: `/v3/projects/1dbee72ef08445e68eab80dd266d2794/users/dc17643051f842fa9e8b49228c3083a5/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:02.030450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ff4b6469a147c1ba753d2b71fbce5c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.031522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ff4b6469a147c1ba753d2b71fbce5c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.069520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ff4b6469a147c1ba753d2b71fbce5c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.070555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ff4b6469a147c1ba753d2b71fbce5c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.073272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d472fb-3c3a-4e00-9982-a0c611573f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dbee72ef08445e68eab80dd266d2794, user_id=dc17643051f842fa9e8b49228c3083a5, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:02.076100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d472fb-3c3a-4e00-9982-a0c611573f3b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:02.104400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 873/1748] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:01 2022] DELETE /identity/v3/users/69ff4b6469a147c1ba753d2b71fbce5c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:02.123212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 876/1749] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:01 2022] PUT /identity/v3/projects/1dbee72ef08445e68eab80dd266d2794/users/dc17643051f842fa9e8b49228c3083a5/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:02.126901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:02.129180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:02.249242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d472fb-3c3a-4e00-9982-a0c611573f3b admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:02.251631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e1d472fb-3c3a-4e00-9982-a0c611573f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:02.272132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe42b5e-580f-481d-9276-3572de8e6181 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:02.450615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d472fb-3c3a-4e00-9982-a0c611573f3b admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:02.469500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:02.473173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:02.476892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:02.477312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] PATH_INFO: `/v3/projects/90153183d10f495486fe9a7bd35283fc` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:02.483960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=90153183d10f495486fe9a7bd35283fc)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:02.491495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba6cf4-a7b9-4f4a-b470-3c9a96aad59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:02.491898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:02.492726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba6cf4-a7b9-4f4a-b470-3c9a96aad59a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:02.493802 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba6cf4-a7b9-4f4a-b470-3c9a96aad59a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:02.497027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba6cf4-a7b9-4f4a-b470-3c9a96aad59a admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:02.499261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba6cf4-a7b9-4f4a-b470-3c9a96aad59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:02.502546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba6cf4-a7b9-4f4a-b470-3c9a96aad59a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:02.516960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:02.524595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 874/1750] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:02.527393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba6cf4-a7b9-4f4a-b470-3c9a96aad59a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:02.535090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-caba6cf4-a7b9-4f4a-b470-3c9a96aad59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:02.575752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba6cf4-a7b9-4f4a-b470-3c9a96aad59a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:02.587434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:02.589520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:02.590170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:02.590703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] PATH_INFO: `/v3/projects/3a0a5970f9994cc690afeb48cecb4b04` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:02.592546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a0a5970f9994cc690afeb48cecb4b04)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:02.595152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:02.689941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90153183d10f495486fe9a7bd35283fc', 'request_id': 'req-4055919c-a233-4682-a1d8-f3c016aa8b9f'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.694868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:02.722661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90153183d10f495486fe9a7bd35283fc', 'request_id': 'req-4055919c-a233-4682-a1d8-f3c016aa8b9f'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.723577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a0a5970f9994cc690afeb48cecb4b04', 'request_id': 'req-917f8882-2e9c-4414-ba30-99a73541668d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.738278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90153183d10f495486fe9a7bd35283fc', 'request_id': 'req-4055919c-a233-4682-a1d8-f3c016aa8b9f'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.759107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a0a5970f9994cc690afeb48cecb4b04', 'request_id': 'req-917f8882-2e9c-4414-ba30-99a73541668d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.771721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a0a5970f9994cc690afeb48cecb4b04', 'request_id': 'req-917f8882-2e9c-4414-ba30-99a73541668d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.783151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] The token cache is being invalidate because project 90153183d10f495486fe9a7bd35283fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:02.783670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.784134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.786963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 877/1751] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:02 2022] DELETE /identity/v3/projects/90153183d10f495486fe9a7bd35283fc => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:02.789811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:02.789811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:02.801443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] The token cache is being invalidate because project 3a0a5970f9994cc690afeb48cecb4b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:02.802386 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.803237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-917f8882-2e9c-4414-ba30-99a73541668d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:02.811824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 875/1752] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:02 2022] DELETE /identity/v3/projects/3a0a5970f9994cc690afeb48cecb4b04 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:02.814248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd077127-e8a1-4d16-996c-173d8d4d4870 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:02.814712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd077127-e8a1-4d16-996c-173d8d4d4870 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:02.815046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd077127-e8a1-4d16-996c-173d8d4d4870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:02.854072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-dd077127-e8a1-4d16-996c-173d8d4d4870 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:02.867653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-dd077127-e8a1-4d16-996c-173d8d4d4870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:03.020829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055919c-a233-4682-a1d8-f3c016aa8b9f admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:03.035781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 876/1753] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:03.036374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50726c3-5ccb-4474-9ec6-196fa5b35f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:03.036858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd077127-e8a1-4d16-996c-173d8d4d4870 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:03.037776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50726c3-5ccb-4474-9ec6-196fa5b35f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:03.038242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50726c3-5ccb-4474-9ec6-196fa5b35f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:03.038349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-dd077127-e8a1-4d16-996c-173d8d4d4870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:03.038824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50726c3-5ccb-4474-9ec6-196fa5b35f91 admin admin] PATH_INFO: `/v3/projects/1dbee72ef08445e68eab80dd266d2794/users/dc17643051f842fa9e8b49228c3083a5/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:03.042281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50726c3-5ccb-4474-9ec6-196fa5b35f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dbee72ef08445e68eab80dd266d2794, user_id=dc17643051f842fa9e8b49228c3083a5, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:03.045399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50726c3-5ccb-4474-9ec6-196fa5b35f91 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:03.057685 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd077127-e8a1-4d16-996c-173d8d4d4870 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:03.109762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 878/1754] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:02 2022] PUT /identity/v3/projects/1dbee72ef08445e68eab80dd266d2794/users/dc17643051f842fa9e8b49228c3083a5/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:03.117405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6c72d-b533-4a24-b129-0ab5ad0bdb87 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:03.120257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6c72d-b533-4a24-b129-0ab5ad0bdb87 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:03.120549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6c72d-b533-4a24-b129-0ab5ad0bdb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:03.191965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-5ed6c72d-b533-4a24-b129-0ab5ad0bdb87 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:03.195401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5ed6c72d-b533-4a24-b129-0ab5ad0bdb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:03.238428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:03.239344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:03.240294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:03.245176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] PATH_INFO: `/v3/users/646afa0e8f0149c1bcd95660fa1e3ebb` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:03.246885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=646afa0e8f0149c1bcd95660fa1e3ebb)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:03.267457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:03.377989 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:03.419205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 879/1755] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:03.419205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb30fc3-9cb9-462f-8e75-1da1ac0f7407 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:03.419205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb30fc3-9cb9-462f-8e75-1da1ac0f7407 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:03.419205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb30fc3-9cb9-462f-8e75-1da1ac0f7407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:03.454584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646afa0e8f0149c1bcd95660fa1e3ebb'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:03.491075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646afa0e8f0149c1bcd95660fa1e3ebb'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:03.492185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646afa0e8f0149c1bcd95660fa1e3ebb'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:03.522815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9eb30fc3-9cb9-462f-8e75-1da1ac0f7407 None None] MFA Rules not processed for user `dc17643051f842fa9e8b49228c3083a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:03.524074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9eb30fc3-9cb9-462f-8e75-1da1ac0f7407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:03.542166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646afa0e8f0149c1bcd95660fa1e3ebb'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:03.543807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-25238c18-a24f-4057-90df-ab7049660f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646afa0e8f0149c1bcd95660fa1e3ebb'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:03.583373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 877/1756] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:03 2022] DELETE /identity/v3/users/646afa0e8f0149c1bcd95660fa1e3ebb => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:03.586440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70de25-f807-42b7-a6ec-be7d591b15c0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:03.587234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70de25-f807-42b7-a6ec-be7d591b15c0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:03.588111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70de25-f807-42b7-a6ec-be7d591b15c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:03.633799 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ce70de25-f807-42b7-a6ec-be7d591b15c0 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:03.642048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ce70de25-f807-42b7-a6ec-be7d591b15c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:03.695247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:03.716293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 880/1757] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:03.717656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb30fc3-9cb9-462f-8e75-1da1ac0f7407 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:03.724077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9eb30fc3-9cb9-462f-8e75-1da1ac0f7407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:03.778822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb30fc3-9cb9-462f-8e75-1da1ac0f7407 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:03.789284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 878/1758] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:03.802603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70de25-f807-42b7-a6ec-be7d591b15c0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:03.806523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ce70de25-f807-42b7-a6ec-be7d591b15c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:04.003376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70de25-f807-42b7-a6ec-be7d591b15c0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:04.015372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfc526-348a-470b-9482-c4eef8525f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:04.017266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfc526-348a-470b-9482-c4eef8525f44 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:04.019317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfc526-348a-470b-9482-c4eef8525f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:04.019804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfc526-348a-470b-9482-c4eef8525f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:04.062725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772a121-e576-452b-9f0d-9dc2b18aa050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:04.063641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772a121-e576-452b-9f0d-9dc2b18aa050 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:04.063796 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772a121-e576-452b-9f0d-9dc2b18aa050 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:04.063956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772a121-e576-452b-9f0d-9dc2b18aa050 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:04.064702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772a121-e576-452b-9f0d-9dc2b18aa050 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:04.065006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772a121-e576-452b-9f0d-9dc2b18aa050 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:04.066666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772a121-e576-452b-9f0d-9dc2b18aa050 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:04.132781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 881/1759] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:04.143546 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772a121-e576-452b-9f0d-9dc2b18aa050 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:04.170300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfc526-348a-470b-9482-c4eef8525f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:04.172303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfc526-348a-470b-9482-c4eef8525f44 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:04.178434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:04.191889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 879/1760] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:04.192767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772a121-e576-452b-9f0d-9dc2b18aa050 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:04.193112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfc526-348a-470b-9482-c4eef8525f44 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:04.195487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8fcfc526-348a-470b-9482-c4eef8525f44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:04.219708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3a0c3-c264-47b6-aa9c-2770cbda26a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:04.226267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a0c3-c264-47b6-aa9c-2770cbda26a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:04.226579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a0c3-c264-47b6-aa9c-2770cbda26a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:04.226864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3a0c3-c264-47b6-aa9c-2770cbda26a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:04.227834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3a0c3-c264-47b6-aa9c-2770cbda26a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:04.238934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3a0c3-c264-47b6-aa9c-2770cbda26a3 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:04.327847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfc526-348a-470b-9482-c4eef8525f44 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:04.337046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b659c-ddb6-4276-8691-66571903e127 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:04.348233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b659c-ddb6-4276-8691-66571903e127 service swift] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:04.348940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b659c-ddb6-4276-8691-66571903e127 service swift] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:04.349387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b659c-ddb6-4276-8691-66571903e127 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:04.358347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 882/1761] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:04 2022] POST /identity/v3/projects => generated 365 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:04.359281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3a0c3-c264-47b6-aa9c-2770cbda26a3 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:04.365277 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b659c-ddb6-4276-8691-66571903e127 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:04.369487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-afe3a0c3-c264-47b6-aa9c-2770cbda26a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:04.370431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b659c-ddb6-4276-8691-66571903e127 service swift] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:04.380857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 880/1762] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:53:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:53:04.382314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b659c-ddb6-4276-8691-66571903e127 service swift] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:04.403499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b659c-ddb6-4276-8691-66571903e127 service swift] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:04.605716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3a0c3-c264-47b6-aa9c-2770cbda26a3 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:04.624567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:04.625602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:04.625901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:04.626188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] PATH_INFO: `/v3/users/4d12ea89fdfc454293a3c2b5e03e2dc3` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:04.636735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d12ea89fdfc454293a3c2b5e03e2dc3)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:04.642446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:04.723369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b07b7-f1a2-4194-a21e-f04a52d98112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:04.728394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b07b7-f1a2-4194-a21e-f04a52d98112 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:04.731610 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b07b7-f1a2-4194-a21e-f04a52d98112 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:04.733197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b07b7-f1a2-4194-a21e-f04a52d98112 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:04.736379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b07b7-f1a2-4194-a21e-f04a52d98112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:04.745218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b07b7-f1a2-4194-a21e-f04a52d98112 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:04.773998 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d12ea89fdfc454293a3c2b5e03e2dc3'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:04.808577 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d12ea89fdfc454293a3c2b5e03e2dc3'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:04.840962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d12ea89fdfc454293a3c2b5e03e2dc3'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:04.842669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:04.843701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 881/1763] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:04 2022] POST /identity/v3/users => generated 283 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:04.847917 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b07b7-f1a2-4194-a21e-f04a52d98112 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:04.866484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d12ea89fdfc454293a3c2b5e03e2dc3'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:04.868240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-b4b7c555-7f76-4410-ab16-aee3120dd5eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d12ea89fdfc454293a3c2b5e03e2dc3'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:04.876232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b07b7-f1a2-4194-a21e-f04a52d98112 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:04.880309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 883/1764] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:04 2022] DELETE /identity/v3/users/4d12ea89fdfc454293a3c2b5e03e2dc3 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:04.887037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec554a9f-3636-4611-86b7-23d972b80f63 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:04.889342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec554a9f-3636-4611-86b7-23d972b80f63 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:04.889638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec554a9f-3636-4611-86b7-23d972b80f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:04.901380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf8390-dcab-4181-b5ca-d5397e3004ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:04.908264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf8390-dcab-4181-b5ca-d5397e3004ca admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:04.908554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf8390-dcab-4181-b5ca-d5397e3004ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:04.908838 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf8390-dcab-4181-b5ca-d5397e3004ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:04.916433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf8390-dcab-4181-b5ca-d5397e3004ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:04.918502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf8390-dcab-4181-b5ca-d5397e3004ca admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:04.987769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 882/1765] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:04.991704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceee000-d838-4645-9bcf-eaeb215664ef None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:04.991998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceee000-d838-4645-9bcf-eaeb215664ef None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:04.995064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceee000-d838-4645-9bcf-eaeb215664ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:05.004536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ec554a9f-3636-4611-86b7-23d972b80f63 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:05.005841 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ec554a9f-3636-4611-86b7-23d972b80f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:05.088298 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-2ceee000-d838-4645-9bcf-eaeb215664ef None None] MFA Rules not processed for user `3198a63f6d1a4cdd88d76189658e79c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:05.093225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2ceee000-d838-4645-9bcf-eaeb215664ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:05.226451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 884/1766] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:05.226451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec554a9f-3636-4611-86b7-23d972b80f63 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:05.227302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ec554a9f-3636-4611-86b7-23d972b80f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:05.243597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec554a9f-3636-4611-86b7-23d972b80f63 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:05.250089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae26645-fbc5-4cdf-8a32-a3648ea37ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:05.261783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae26645-fbc5-4cdf-8a32-a3648ea37ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:05.262187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae26645-fbc5-4cdf-8a32-a3648ea37ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:05.262470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae26645-fbc5-4cdf-8a32-a3648ea37ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:05.268912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae26645-fbc5-4cdf-8a32-a3648ea37ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:05.271223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae26645-fbc5-4cdf-8a32-a3648ea37ff4 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:05.298302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 885/1767] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:05.302228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae26645-fbc5-4cdf-8a32-a3648ea37ff4 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:05.306531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9ae26645-fbc5-4cdf-8a32-a3648ea37ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:05.353000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 883/1768] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:05.359003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceee000-d838-4645-9bcf-eaeb215664ef None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:05.415140 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceee000-d838-4645-9bcf-eaeb215664ef None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:05.440656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74197fd-0d13-4ab9-8e7a-ec04fe204ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:05.442472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74197fd-0d13-4ab9-8e7a-ec04fe204ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:05.449390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74197fd-0d13-4ab9-8e7a-ec04fe204ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:05.449899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74197fd-0d13-4ab9-8e7a-ec04fe204ac2 admin admin] PATH_INFO: `/v3/projects/2bc78a09cde84f10b2c33fa7a2332f92/users/5543933aa25b4dccb845ca336bf45dea/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:05.471333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:05.480800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae26645-fbc5-4cdf-8a32-a3648ea37ff4 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:05.488974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf459b39-cf14-4210-8d91-78684fcf532c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:05.489663 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf459b39-cf14-4210-8d91-78684fcf532c service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:05.489950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf459b39-cf14-4210-8d91-78684fcf532c service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:05.490226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf459b39-cf14-4210-8d91-78684fcf532c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:05.491332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74197fd-0d13-4ab9-8e7a-ec04fe204ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bc78a09cde84f10b2c33fa7a2332f92, user_id=5543933aa25b4dccb845ca336bf45dea, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:05.497041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74197fd-0d13-4ab9-8e7a-ec04fe204ac2 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:05.519182 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf459b39-cf14-4210-8d91-78684fcf532c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:05.520704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf459b39-cf14-4210-8d91-78684fcf532c service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:05.535061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 884/1769] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:05 2022] PUT /identity/v3/projects/2bc78a09cde84f10b2c33fa7a2332f92/users/5543933aa25b4dccb845ca336bf45dea/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:05.546685 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df4d8-854c-4a0c-a74a-bb8808013c86 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:05.546974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df4d8-854c-4a0c-a74a-bb8808013c86 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:05.547254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df4d8-854c-4a0c-a74a-bb8808013c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:05.556985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 886/1770] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:05.640810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-501df4d8-854c-4a0c-a74a-bb8808013c86 None None] MFA Rules not processed for user `5543933aa25b4dccb845ca336bf45dea`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:05.642293 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-501df4d8-854c-4a0c-a74a-bb8808013c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:05.815405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 885/1771] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:05.817204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf459b39-cf14-4210-8d91-78684fcf532c service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:05.854001 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf459b39-cf14-4210-8d91-78684fcf532c service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:05.999331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95faf247-cb17-40b8-928b-345d40615943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:06.000013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faf247-cb17-40b8-928b-345d40615943 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:06.000297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faf247-cb17-40b8-928b-345d40615943 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:06.000573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faf247-cb17-40b8-928b-345d40615943 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:06.001277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95faf247-cb17-40b8-928b-345d40615943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:06.003520 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95faf247-cb17-40b8-928b-345d40615943 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:06.140899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 887/1772] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:05 2022] POST /identity/v3/projects => generated 363 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:06.145206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501df4d8-854c-4a0c-a74a-bb8808013c86 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:06.184448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95faf247-cb17-40b8-928b-345d40615943 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:06.187061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-95faf247-cb17-40b8-928b-345d40615943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:06.202175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:06.204753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501df4d8-854c-4a0c-a74a-bb8808013c86 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:06.385194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95faf247-cb17-40b8-928b-345d40615943 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:06.394203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:06.395451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:06.395962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:06.396449 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] PATH_INFO: `/v3/projects/6d8b1593aab64207929471dcb695c0d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:06.402369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8b1593aab64207929471dcb695c0d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:06.407808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:06.449526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461742f-512f-4ca8-be8e-51ff49b21f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:06.450257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461742f-512f-4ca8-be8e-51ff49b21f55 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:06.450545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461742f-512f-4ca8-be8e-51ff49b21f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:06.450819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461742f-512f-4ca8-be8e-51ff49b21f55 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:06.451535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461742f-512f-4ca8-be8e-51ff49b21f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:06.453531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461742f-512f-4ca8-be8e-51ff49b21f55 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:06.514143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 886/1773] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:06 2022] POST /identity/v3/users => generated 288 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:53:06.515429 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461742f-512f-4ca8-be8e-51ff49b21f55 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:06.520648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a461742f-512f-4ca8-be8e-51ff49b21f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:06.554152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8b1593aab64207929471dcb695c0d5', 'request_id': 'req-8316af75-ff01-4dd2-b7de-7038e89dbaff'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:06.557638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461742f-512f-4ca8-be8e-51ff49b21f55 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:06.567929 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8b1593aab64207929471dcb695c0d5', 'request_id': 'req-8316af75-ff01-4dd2-b7de-7038e89dbaff'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:06.581870 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8b1593aab64207929471dcb695c0d5', 'request_id': 'req-8316af75-ff01-4dd2-b7de-7038e89dbaff'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:06.616285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] The token cache is being invalidate because project 6d8b1593aab64207929471dcb695c0d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:06.616880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:06.618073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:06.621017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 888/1774] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:06 2022] DELETE /identity/v3/projects/6d8b1593aab64207929471dcb695c0d5 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:06.622344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:06.628465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:06.746877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:06.748755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:06.749256 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:06.750693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] PATH_INFO: `/v3/projects/ba1ea012747e4a36badacb6d401c3735` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:06.753320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba1ea012747e4a36badacb6d401c3735)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:06.764195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:06.764807 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316af75-ff01-4dd2-b7de-7038e89dbaff admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:06.770089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd5fad-a051-421f-9fab-a81a773b6055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:06.770767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd5fad-a051-421f-9fab-a81a773b6055 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:06.771053 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd5fad-a051-421f-9fab-a81a773b6055 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:06.771325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd5fad-a051-421f-9fab-a81a773b6055 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:06.772248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd5fad-a051-421f-9fab-a81a773b6055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:06.774148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd5fad-a051-421f-9fab-a81a773b6055 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:06.791865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 889/1775] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:06.792913 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd5fad-a051-421f-9fab-a81a773b6055 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:06.795140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-31dd5fad-a051-421f-9fab-a81a773b6055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:06.796285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:06.887498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd5fad-a051-421f-9fab-a81a773b6055 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:06.937912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba1ea012747e4a36badacb6d401c3735', 'request_id': 'req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:06.956330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba1ea012747e4a36badacb6d401c3735', 'request_id': 'req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:06.971489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba1ea012747e4a36badacb6d401c3735', 'request_id': 'req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:07.006387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] The token cache is being invalidate because project ba1ea012747e4a36badacb6d401c3735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:07.006954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:07.008305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2dcc64da-ba74-43f2-ba68-0d02ea8743ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:07.019075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 887/1776] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:06 2022] DELETE /identity/v3/projects/ba1ea012747e4a36badacb6d401c3735 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:07.021962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96476-de7e-4158-999c-50c8657a4fd7 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:07.022708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96476-de7e-4158-999c-50c8657a4fd7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:07.023679 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96476-de7e-4158-999c-50c8657a4fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:07.049477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:07.071654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c6e96476-de7e-4158-999c-50c8657a4fd7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:07.078280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c6e96476-de7e-4158-999c-50c8657a4fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:07.190559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:07.191676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:07.191965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:07.192286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] PATH_INFO: `/v3/projects/90956bebf5a44cb4a214106996e50db3` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:07.195965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90956bebf5a44cb4a214106996e50db3)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:07.207266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:07.270454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 888/1777] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:07.301142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96476-de7e-4158-999c-50c8657a4fd7 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:07.303035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c6e96476-de7e-4158-999c-50c8657a4fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:07.335175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96476-de7e-4158-999c-50c8657a4fd7 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:07.398959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90956bebf5a44cb4a214106996e50db3', 'request_id': 'req-21b70833-3a00-40f2-9c04-58171a43d819'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:07.432274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90956bebf5a44cb4a214106996e50db3', 'request_id': 'req-21b70833-3a00-40f2-9c04-58171a43d819'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:07.444606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90956bebf5a44cb4a214106996e50db3', 'request_id': 'req-21b70833-3a00-40f2-9c04-58171a43d819'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:07.484893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] The token cache is being invalidate because project 90956bebf5a44cb4a214106996e50db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:07.485347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:07.487095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:07.496096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 890/1778] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:06 2022] DELETE /identity/v3/projects/90956bebf5a44cb4a214106996e50db3 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:07.496096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:07.498585 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:07.536595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69c45b-16b9-461e-ba7d-2545f28a5e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:07.541471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69c45b-16b9-461e-ba7d-2545f28a5e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:07.542225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69c45b-16b9-461e-ba7d-2545f28a5e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:07.543758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69c45b-16b9-461e-ba7d-2545f28a5e6f admin admin] PATH_INFO: `/v3/projects/22623461dc0245a5afbda0f49e80f161/users/c9bedf982994494e87cee97d97e96f83/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:07.621042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69c45b-16b9-461e-ba7d-2545f28a5e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=22623461dc0245a5afbda0f49e80f161, user_id=c9bedf982994494e87cee97d97e96f83, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:07.630809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69c45b-16b9-461e-ba7d-2545f28a5e6f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:07.675352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 889/1779] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:07 2022] PUT /identity/v3/projects/22623461dc0245a5afbda0f49e80f161/users/c9bedf982994494e87cee97d97e96f83/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:07.690667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bb016-7215-431d-bf61-0935ecfc5a66 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:07.694393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bb016-7215-431d-bf61-0935ecfc5a66 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:07.694699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bb016-7215-431d-bf61-0935ecfc5a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:07.782576 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-045bb016-7215-431d-bf61-0935ecfc5a66 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:07.786580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-045bb016-7215-431d-bf61-0935ecfc5a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:07.787703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b70833-3a00-40f2-9c04-58171a43d819 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:07.822945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:07.953670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:07.954802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:07.955240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:07.955528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] PATH_INFO: `/v3/users/dc17643051f842fa9e8b49228c3083a5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:07.960201 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc17643051f842fa9e8b49228c3083a5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:07.963783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:07.998090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:08.104795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc17643051f842fa9e8b49228c3083a5'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:08.106290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 890/1780] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:08.125281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dd799-06fc-4663-ad28-bf67203c33cc None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:08.125603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dd799-06fc-4663-ad28-bf67203c33cc None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:08.138575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dd799-06fc-4663-ad28-bf67203c33cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:08.190445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc17643051f842fa9e8b49228c3083a5'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:08.234247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc17643051f842fa9e8b49228c3083a5'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:08.265994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-8a7dd799-06fc-4663-ad28-bf67203c33cc None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:08.268078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8a7dd799-06fc-4663-ad28-bf67203c33cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:08.274282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc17643051f842fa9e8b49228c3083a5'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:08.278587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc17643051f842fa9e8b49228c3083a5'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:08.307670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 891/1781] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:07 2022] DELETE /identity/v3/users/dc17643051f842fa9e8b49228c3083a5 => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:08.308677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:08.317790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 891/1782] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:08.317790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dd799-06fc-4663-ad28-bf67203c33cc None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:08.318415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:08.319765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8a7dd799-06fc-4663-ad28-bf67203c33cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:08.394728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dd799-06fc-4663-ad28-bf67203c33cc None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:08.657439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3450a-f2c1-4830-8832-8a3663d521c5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:08.676109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4b79e-379e-4c1c-8262-3fedb0a0136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:08.679336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4b79e-379e-4c1c-8262-3fedb0a0136b admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:08.682590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4b79e-379e-4c1c-8262-3fedb0a0136b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:08.683049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdced3-0b87-465b-a083-0ee073d1b18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:08.683469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4b79e-379e-4c1c-8262-3fedb0a0136b admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:08.684254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdced3-0b87-465b-a083-0ee073d1b18e admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:08.689561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4b79e-379e-4c1c-8262-3fedb0a0136b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:08.690116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdced3-0b87-465b-a083-0ee073d1b18e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:08.690497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdced3-0b87-465b-a083-0ee073d1b18e admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:08.691830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4b79e-379e-4c1c-8262-3fedb0a0136b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:08.694690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4b79e-379e-4c1c-8262-3fedb0a0136b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:08.695014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdced3-0b87-465b-a083-0ee073d1b18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:08.702605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdced3-0b87-465b-a083-0ee073d1b18e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:08.739393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 892/1783] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:08.755199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffa781-fa6b-478c-a22a-fbac97bcc7ea None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:08.756484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffa781-fa6b-478c-a22a-fbac97bcc7ea None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:08.756933 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffa781-fa6b-478c-a22a-fbac97bcc7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:08.761253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 892/1784] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:08.770494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdced3-0b87-465b-a083-0ee073d1b18e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:08.804348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdced3-0b87-465b-a083-0ee073d1b18e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:08.836572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da4067-cd4b-4df2-a6f0-444e49aa7958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:08.838646 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da4067-cd4b-4df2-a6f0-444e49aa7958 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:08.839269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da4067-cd4b-4df2-a6f0-444e49aa7958 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:08.839762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da4067-cd4b-4df2-a6f0-444e49aa7958 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:08.840885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da4067-cd4b-4df2-a6f0-444e49aa7958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:08.843088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da4067-cd4b-4df2-a6f0-444e49aa7958 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:08.888177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-96ffa781-fa6b-478c-a22a-fbac97bcc7ea None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:08.897888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-96ffa781-fa6b-478c-a22a-fbac97bcc7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:08.924597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 893/1785] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:08 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:08.926634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da4067-cd4b-4df2-a6f0-444e49aa7958 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:08.955775 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 893/1786] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:08.963579 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffa781-fa6b-478c-a22a-fbac97bcc7ea None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:09.003388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffa781-fa6b-478c-a22a-fbac97bcc7ea None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:09.175227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da4067-cd4b-4df2-a6f0-444e49aa7958 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:09.193650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a19c-fc73-496b-bc24-4fc9b7e99cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:09.195230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a19c-fc73-496b-bc24-4fc9b7e99cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:09.195522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a19c-fc73-496b-bc24-4fc9b7e99cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:09.195799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a19c-fc73-496b-bc24-4fc9b7e99cbf admin admin] PATH_INFO: `/v3/projects/22623461dc0245a5afbda0f49e80f161/users/c9bedf982994494e87cee97d97e96f83/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:09.201128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22a19c-fc73-496b-bc24-4fc9b7e99cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=22623461dc0245a5afbda0f49e80f161, user_id=c9bedf982994494e87cee97d97e96f83, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:09.203887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22a19c-fc73-496b-bc24-4fc9b7e99cbf admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:09.233818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:09.260297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 894/1787] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:08 2022] PUT /identity/v3/projects/22623461dc0245a5afbda0f49e80f161/users/c9bedf982994494e87cee97d97e96f83/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:09.263838 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a19c-fc73-496b-bc24-4fc9b7e99cbf admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:09.307049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0e67f-1c33-4c8a-be77-e23d425b03ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:09.310869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0e67f-1c33-4c8a-be77-e23d425b03ae admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:09.311160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0e67f-1c33-4c8a-be77-e23d425b03ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:09.311537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0e67f-1c33-4c8a-be77-e23d425b03ae admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:09.312257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0e67f-1c33-4c8a-be77-e23d425b03ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:09.326359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0e67f-1c33-4c8a-be77-e23d425b03ae admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:09.395287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 894/1788] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:08 2022] POST /identity/v3/users => generated 284 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:09.396755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525501-f6dd-411d-81bd-c70425fe7f9a None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:09.397067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525501-f6dd-411d-81bd-c70425fe7f9a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:09.397366 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525501-f6dd-411d-81bd-c70425fe7f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:09.519901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-98525501-f6dd-411d-81bd-c70425fe7f9a None None] MFA Rules not processed for user `c9bedf982994494e87cee97d97e96f83`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:09.522946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-98525501-f6dd-411d-81bd-c70425fe7f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:09.524436 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a19c-fc73-496b-bc24-4fc9b7e99cbf admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:09.546245 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6488a8-9058-435e-a864-fb744902c838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:09.551786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6488a8-9058-435e-a864-fb744902c838 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:09.552073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6488a8-9058-435e-a864-fb744902c838 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:09.552355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6488a8-9058-435e-a864-fb744902c838 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:09.554100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6488a8-9058-435e-a864-fb744902c838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:09.554420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6488a8-9058-435e-a864-fb744902c838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:09.556166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6488a8-9058-435e-a864-fb744902c838 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:09.605064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 895/1789] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:09.606919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6488a8-9058-435e-a864-fb744902c838 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:09.615652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:09.630867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6488a8-9058-435e-a864-fb744902c838 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:09.645933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e203afc-a031-4005-888e-8f7700b6f35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:09.647317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203afc-a031-4005-888e-8f7700b6f35e admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:09.647764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203afc-a031-4005-888e-8f7700b6f35e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:09.648099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203afc-a031-4005-888e-8f7700b6f35e admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:09.651483 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e203afc-a031-4005-888e-8f7700b6f35e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:09.655315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e203afc-a031-4005-888e-8f7700b6f35e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:09.679689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 896/1790] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:09.679689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e203afc-a031-4005-888e-8f7700b6f35e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:09.726342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e203afc-a031-4005-888e-8f7700b6f35e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:09.735630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ff315-2a47-4100-b6bb-d2a359ea21b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:09.736571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ff315-2a47-4100-b6bb-d2a359ea21b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:09.750099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ff315-2a47-4100-b6bb-d2a359ea21b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:09.750821 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ff315-2a47-4100-b6bb-d2a359ea21b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:09.753527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ff315-2a47-4100-b6bb-d2a359ea21b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:09.758542 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ff315-2a47-4100-b6bb-d2a359ea21b6 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:09.759090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 895/1791] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:09.760632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98525501-f6dd-411d-81bd-c70425fe7f9a None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:09.787580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98525501-f6dd-411d-81bd-c70425fe7f9a None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:09.810521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bac61c-b3d9-4489-8776-284b0c42efce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:09.811231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bac61c-b3d9-4489-8776-284b0c42efce admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:09.811513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bac61c-b3d9-4489-8776-284b0c42efce admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:09.811788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bac61c-b3d9-4489-8776-284b0c42efce admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:09.812720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bac61c-b3d9-4489-8776-284b0c42efce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:09.814919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bac61c-b3d9-4489-8776-284b0c42efce admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:09.850844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 896/1792] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:09.852050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bac61c-b3d9-4489-8776-284b0c42efce admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:09.856962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-44bac61c-b3d9-4489-8776-284b0c42efce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:09.908115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 897/1793] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:09 2022] POST /identity/v3/projects => generated 381 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:09.910990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ff315-2a47-4100-b6bb-d2a359ea21b6 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:10.114312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bac61c-b3d9-4489-8776-284b0c42efce admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:10.170292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ff315-2a47-4100-b6bb-d2a359ea21b6 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:10.215140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8eabb8-3b7c-41ca-a4e4-19b4f8b27a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:10.217300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8eabb8-3b7c-41ca-a4e4-19b4f8b27a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:10.217628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8eabb8-3b7c-41ca-a4e4-19b4f8b27a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:10.218433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8eabb8-3b7c-41ca-a4e4-19b4f8b27a8c admin admin] PATH_INFO: `/v3/projects/f97e11323fba4bd6ba4f55b97a3b14e4/users/ed9df1a4183446af94d57f655443b77c/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:10.278503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16876f-46b7-4b35-bd35-3533a76ee5df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:10.280615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16876f-46b7-4b35-bd35-3533a76ee5df service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:10.281009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16876f-46b7-4b35-bd35-3533a76ee5df service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:10.281292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16876f-46b7-4b35-bd35-3533a76ee5df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:10.297484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8eabb8-3b7c-41ca-a4e4-19b4f8b27a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97e11323fba4bd6ba4f55b97a3b14e4, user_id=ed9df1a4183446af94d57f655443b77c, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:10.314133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8eabb8-3b7c-41ca-a4e4-19b4f8b27a8c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:10.344289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 898/1794] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:09 2022] PUT /identity/v3/projects/f97e11323fba4bd6ba4f55b97a3b14e4/users/ed9df1a4183446af94d57f655443b77c/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:10.345279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8eabb8-3b7c-41ca-a4e4-19b4f8b27a8c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:10.391952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16876f-46b7-4b35-bd35-3533a76ee5df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:10.393710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16876f-46b7-4b35-bd35-3533a76ee5df service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:10.406892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 897/1795] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:09 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:10.408464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dacedc-5021-4cfc-9a98-78c90777d165 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:10.408762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dacedc-5021-4cfc-9a98-78c90777d165 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:10.409041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dacedc-5021-4cfc-9a98-78c90777d165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:10.439783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-08dacedc-5021-4cfc-9a98-78c90777d165 None None] MFA Rules not processed for user `370f1c0f7af04a56bdf8fa452550380a`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:10.441039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-08dacedc-5021-4cfc-9a98-78c90777d165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:10.448668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:10.475427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:10.530025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 898/1796] 10.222.0.60 () {58 vars in 1015 bytes} [Fri Jul 1 19:53:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:10.532500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42432-45c1-4520-b843-828ce084421e None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:10.532869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42432-45c1-4520-b843-828ce084421e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:10.533315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab42432-45c1-4520-b843-828ce084421e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:10.544336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8eabb8-3b7c-41ca-a4e4-19b4f8b27a8c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:10.578491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44892653-b417-4a85-b315-c0ac97e3d7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:10.579187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44892653-b417-4a85-b315-c0ac97e3d7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:10.579470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44892653-b417-4a85-b315-c0ac97e3d7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:10.579750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44892653-b417-4a85-b315-c0ac97e3d7d1 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:10.580454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44892653-b417-4a85-b315-c0ac97e3d7d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:10.582631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44892653-b417-4a85-b315-c0ac97e3d7d1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:10.612540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-cab42432-45c1-4520-b843-828ce084421e None None] MFA Rules not processed for user `ed9df1a4183446af94d57f655443b77c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:10.614701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-cab42432-45c1-4520-b843-828ce084421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:10.622545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 899/1797] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:10 2022] POST /identity/v3/users => generated 291 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:10.623560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44892653-b417-4a85-b315-c0ac97e3d7d1 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:10.717977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 899/1798] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:10.722574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab42432-45c1-4520-b843-828ce084421e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:10.759163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab42432-45c1-4520-b843-828ce084421e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:10.779081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc3d4c-4bcf-498e-8b9a-88d3e29ee87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:10.780049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc3d4c-4bcf-498e-8b9a-88d3e29ee87d admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:10.784081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc3d4c-4bcf-498e-8b9a-88d3e29ee87d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:10.784452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc3d4c-4bcf-498e-8b9a-88d3e29ee87d admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:10.785381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc3d4c-4bcf-498e-8b9a-88d3e29ee87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:10.787314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc3d4c-4bcf-498e-8b9a-88d3e29ee87d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:10.790984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44892653-b417-4a85-b315-c0ac97e3d7d1 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:10.803916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251f84a-bade-4ba9-a021-87a510af7c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:10.804648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251f84a-bade-4ba9-a021-87a510af7c8c service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:10.804930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251f84a-bade-4ba9-a021-87a510af7c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:10.805207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251f84a-bade-4ba9-a021-87a510af7c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:10.818544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5251f84a-bade-4ba9-a021-87a510af7c8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:10.829566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 900/1799] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:10.830431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc3d4c-4bcf-498e-8b9a-88d3e29ee87d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:10.872119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc3d4c-4bcf-498e-8b9a-88d3e29ee87d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:10.886068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d24e2c-4797-404a-b578-227cb71a62eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:10.886929 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d24e2c-4797-404a-b578-227cb71a62eb admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:10.887216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d24e2c-4797-404a-b578-227cb71a62eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:10.887668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d24e2c-4797-404a-b578-227cb71a62eb admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:10.889072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d24e2c-4797-404a-b578-227cb71a62eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:10.892078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d24e2c-4797-404a-b578-227cb71a62eb admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:10.984268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 901/1800] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:10 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:10.985679 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d24e2c-4797-404a-b578-227cb71a62eb admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:11.004618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251f84a-bade-4ba9-a021-87a510af7c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:11.023199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251f84a-bade-4ba9-a021-87a510af7c8c service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:11.069164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 900/1801] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:11.080785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251f84a-bade-4ba9-a021-87a510af7c8c service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:11.103584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5251f84a-bade-4ba9-a021-87a510af7c8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:11.144863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251f84a-bade-4ba9-a021-87a510af7c8c service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:11.185440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d24e2c-4797-404a-b578-227cb71a62eb admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:11.206992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c4b72-2651-4d50-9813-4d3477f9b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:11.208422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c4b72-2651-4d50-9813-4d3477f9b488 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:11.208422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c4b72-2651-4d50-9813-4d3477f9b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:11.208769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c4b72-2651-4d50-9813-4d3477f9b488 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:11.223746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c4b72-2651-4d50-9813-4d3477f9b488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:11.226895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c4b72-2651-4d50-9813-4d3477f9b488 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:11.268843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 902/1802] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:11.279038 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c4b72-2651-4d50-9813-4d3477f9b488 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:11.286402 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6b8c4b72-2651-4d50-9813-4d3477f9b488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:11.349944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:11.391938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f58ad-971e-4f8c-82fa-1ece5aa687fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:11.392799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f58ad-971e-4f8c-82fa-1ece5aa687fc service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:11.393071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f58ad-971e-4f8c-82fa-1ece5aa687fc service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:11.393358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f58ad-971e-4f8c-82fa-1ece5aa687fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:11.423277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f58ad-971e-4f8c-82fa-1ece5aa687fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:11.434828 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f58ad-971e-4f8c-82fa-1ece5aa687fc service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:11.469791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 901/1803] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:11.470814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f58ad-971e-4f8c-82fa-1ece5aa687fc service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:11.513102 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f58ad-971e-4f8c-82fa-1ece5aa687fc service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:11.538340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735e91c-9b70-4672-b520-49dd5a011b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:11.539128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e91c-9b70-4672-b520-49dd5a011b99 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:11.539399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e91c-9b70-4672-b520-49dd5a011b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:11.539683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735e91c-9b70-4672-b520-49dd5a011b99 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:11.541717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735e91c-9b70-4672-b520-49dd5a011b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:11.554116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735e91c-9b70-4672-b520-49dd5a011b99 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:11.625873 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 902/1804] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:11 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:11.628137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735e91c-9b70-4672-b520-49dd5a011b99 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:11.636113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1735e91c-9b70-4672-b520-49dd5a011b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:11.645272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c4b72-2651-4d50-9813-4d3477f9b488 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:11.652212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb162b-6eb4-4cb3-85c9-e633909b7873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:11.652942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb162b-6eb4-4cb3-85c9-e633909b7873 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:11.653288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb162b-6eb4-4cb3-85c9-e633909b7873 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:11.669718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb162b-6eb4-4cb3-85c9-e633909b7873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:11.680857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb162b-6eb4-4cb3-85c9-e633909b7873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:11.683865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb162b-6eb4-4cb3-85c9-e633909b7873 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:11.685149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735e91c-9b70-4672-b520-49dd5a011b99 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:11.711288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:11.715279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:11.715889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:11.717935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] PATH_INFO: `/v3/projects/1dbee72ef08445e68eab80dd266d2794` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:11.722468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dbee72ef08445e68eab80dd266d2794)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:11.726918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 903/1805] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:11.728743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb162b-6eb4-4cb3-85c9-e633909b7873 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:11.734748 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:11.763881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb162b-6eb4-4cb3-85c9-e633909b7873 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:11.783091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:11.784394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f1389-b941-469e-9175-f4d016eafd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:11.787104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f1389-b941-469e-9175-f4d016eafd46 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:11.789644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f1389-b941-469e-9175-f4d016eafd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:11.790053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f1389-b941-469e-9175-f4d016eafd46 admin admin] PATH_INFO: `/v3/projects/b70f786b912248f7ab167a39bf7c1f70/users/13f50e4374204f36a24268f302304625/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:11.896301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f1389-b941-469e-9175-f4d016eafd46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70f786b912248f7ab167a39bf7c1f70, user_id=13f50e4374204f36a24268f302304625, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:11.912306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f1389-b941-469e-9175-f4d016eafd46 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:12.018527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 904/1806] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:11 2022] PUT /identity/v3/projects/b70f786b912248f7ab167a39bf7c1f70/users/13f50e4374204f36a24268f302304625/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:12.028723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f1389-b941-469e-9175-f4d016eafd46 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:12.083701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dbee72ef08445e68eab80dd266d2794', 'request_id': 'req-a689e5ca-1fc1-4dba-b74f-d685789a5503'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:12.140005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dbee72ef08445e68eab80dd266d2794', 'request_id': 'req-a689e5ca-1fc1-4dba-b74f-d685789a5503'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:12.171306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dbee72ef08445e68eab80dd266d2794', 'request_id': 'req-a689e5ca-1fc1-4dba-b74f-d685789a5503'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:12.232529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] The token cache is being invalidate because project 1dbee72ef08445e68eab80dd266d2794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:12.233990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:12.235090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a689e5ca-1fc1-4dba-b74f-d685789a5503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:12.239086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 903/1807] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:11 2022] DELETE /identity/v3/projects/1dbee72ef08445e68eab80dd266d2794 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:12.240966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abfb9e-6328-4e1e-8724-1f5d26aac16c None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:12.241409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abfb9e-6328-4e1e-8724-1f5d26aac16c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:12.242047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abfb9e-6328-4e1e-8724-1f5d26aac16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:12.311051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c1abfb9e-6328-4e1e-8724-1f5d26aac16c None None] MFA Rules not processed for user `13f50e4374204f36a24268f302304625`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:12.312620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c1abfb9e-6328-4e1e-8724-1f5d26aac16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:12.378978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f1389-b941-469e-9175-f4d016eafd46 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:12.403633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa14708-465b-4549-bb10-d7850c671e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:12.404339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa14708-465b-4549-bb10-d7850c671e48 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:12.404625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa14708-465b-4549-bb10-d7850c671e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:12.404908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa14708-465b-4549-bb10-d7850c671e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:12.405686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa14708-465b-4549-bb10-d7850c671e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:12.424314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa14708-465b-4549-bb10-d7850c671e48 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:12.469313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 905/1808] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:12.477420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6584fd9-98db-4fd6-811e-0622ad28162b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:12.479598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6584fd9-98db-4fd6-811e-0622ad28162b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:12.479910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6584fd9-98db-4fd6-811e-0622ad28162b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:12.491939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 904/1809] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:12.492800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abfb9e-6328-4e1e-8724-1f5d26aac16c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:12.497808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c1abfb9e-6328-4e1e-8724-1f5d26aac16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:12.549817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c6584fd9-98db-4fd6-811e-0622ad28162b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:12.550842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c6584fd9-98db-4fd6-811e-0622ad28162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:12.558627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abfb9e-6328-4e1e-8724-1f5d26aac16c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:12.572377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267aad9e-7a1e-4b21-a492-2f2f397c12b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:12.573916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aad9e-7a1e-4b21-a492-2f2f397c12b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:12.573916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aad9e-7a1e-4b21-a492-2f2f397c12b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:12.573916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aad9e-7a1e-4b21-a492-2f2f397c12b3 admin admin] PATH_INFO: `/v3/projects/bd6646586b4f4fb1938ac90044d81da0/users/9a5ab9e1e14e48a49293840e74e84d15/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:12.588590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 906/1810] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:12.589538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6584fd9-98db-4fd6-811e-0622ad28162b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:12.592706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c6584fd9-98db-4fd6-811e-0622ad28162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:12.620337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267aad9e-7a1e-4b21-a492-2f2f397c12b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6646586b4f4fb1938ac90044d81da0, user_id=9a5ab9e1e14e48a49293840e74e84d15, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:12.628971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267aad9e-7a1e-4b21-a492-2f2f397c12b3 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:12.632902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6584fd9-98db-4fd6-811e-0622ad28162b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:12.658888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4f6f9-e0e9-4984-a0ab-8458343e5738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:12.660500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4f6f9-e0e9-4984-a0ab-8458343e5738 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:12.661088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 905/1811] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:12 2022] PUT /identity/v3/projects/bd6646586b4f4fb1938ac90044d81da0/users/9a5ab9e1e14e48a49293840e74e84d15/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:12.662932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4f6f9-e0e9-4984-a0ab-8458343e5738 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:12.663071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c952f-32de-4839-bb33-bddcb202527f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:12.663224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c952f-32de-4839-bb33-bddcb202527f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:12.664507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c952f-32de-4839-bb33-bddcb202527f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:12.666270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4f6f9-e0e9-4984-a0ab-8458343e5738 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:12.667362 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4f6f9-e0e9-4984-a0ab-8458343e5738 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:12.681427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4f6f9-e0e9-4984-a0ab-8458343e5738 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:12.736579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-fc9c952f-32de-4839-bb33-bddcb202527f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:12.739714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fc9c952f-32de-4839-bb33-bddcb202527f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:12.823678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 907/1812] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:12 2022] POST /identity/v3/projects => generated 381 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:12.823678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4f6f9-e0e9-4984-a0ab-8458343e5738 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:12.855460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:12.868095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 906/1813] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:12.873939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c952f-32de-4839-bb33-bddcb202527f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:12.888853 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c952f-32de-4839-bb33-bddcb202527f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:12.999954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4f6f9-e0e9-4984-a0ab-8458343e5738 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:13.017180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a91a94-8e38-41cf-b6b7-fc34cced9895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:13.028119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91a94-8e38-41cf-b6b7-fc34cced9895 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:13.028375 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91a94-8e38-41cf-b6b7-fc34cced9895 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:13.028976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91a94-8e38-41cf-b6b7-fc34cced9895 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:13.030641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a91a94-8e38-41cf-b6b7-fc34cced9895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:13.033138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a91a94-8e38-41cf-b6b7-fc34cced9895 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:13.038540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:13.082326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 908/1814] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:13.087002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a91a94-8e38-41cf-b6b7-fc34cced9895 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:13.109503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709b477-7b05-49a5-8c22-94f16ea6d3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:13.111728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709b477-7b05-49a5-8c22-94f16ea6d3ae admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:13.112128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709b477-7b05-49a5-8c22-94f16ea6d3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:13.116319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709b477-7b05-49a5-8c22-94f16ea6d3ae admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:13.123002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709b477-7b05-49a5-8c22-94f16ea6d3ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:13.139182 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709b477-7b05-49a5-8c22-94f16ea6d3ae admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:13.142714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a91a94-8e38-41cf-b6b7-fc34cced9895 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:13.181632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3bb58-0507-4deb-bc5d-97cd9bf8e5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:13.190900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3bb58-0507-4deb-bc5d-97cd9bf8e5ad admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:13.192069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3bb58-0507-4deb-bc5d-97cd9bf8e5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:13.194292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3bb58-0507-4deb-bc5d-97cd9bf8e5ad admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:13.195933 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3bb58-0507-4deb-bc5d-97cd9bf8e5ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:13.196376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3bb58-0507-4deb-bc5d-97cd9bf8e5ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:13.199554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3bb58-0507-4deb-bc5d-97cd9bf8e5ad admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:13.229245 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 907/1815] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:12 2022] POST /identity/v3/users => generated 297 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:13.258699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709b477-7b05-49a5-8c22-94f16ea6d3ae admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:13.262772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 909/1816] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:13.263838 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3bb58-0507-4deb-bc5d-97cd9bf8e5ad admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:13.291346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709b477-7b05-49a5-8c22-94f16ea6d3ae admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:13.297083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86239fc6-1fff-4bc5-a580-311b09d6ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:13.301286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86239fc6-1fff-4bc5-a580-311b09d6ae25 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:13.301598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86239fc6-1fff-4bc5-a580-311b09d6ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:13.301891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86239fc6-1fff-4bc5-a580-311b09d6ae25 admin admin] PATH_INFO: `/v3/projects/bd6646586b4f4fb1938ac90044d81da0/users/9a5ab9e1e14e48a49293840e74e84d15/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:13.304812 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86239fc6-1fff-4bc5-a580-311b09d6ae25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6646586b4f4fb1938ac90044d81da0, user_id=9a5ab9e1e14e48a49293840e74e84d15, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:13.308292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3bb58-0507-4deb-bc5d-97cd9bf8e5ad admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:13.308886 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86239fc6-1fff-4bc5-a580-311b09d6ae25 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:13.334333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8af33-2069-4d65-987a-cf86b6dd96cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:13.335082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8af33-2069-4d65-987a-cf86b6dd96cb admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:13.335418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8af33-2069-4d65-987a-cf86b6dd96cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:13.335607 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8af33-2069-4d65-987a-cf86b6dd96cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:13.336357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8af33-2069-4d65-987a-cf86b6dd96cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:13.345746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8af33-2069-4d65-987a-cf86b6dd96cb admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:13.362753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 908/1817] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:13 2022] PUT /identity/v3/projects/bd6646586b4f4fb1938ac90044d81da0/users/9a5ab9e1e14e48a49293840e74e84d15/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:13.366081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86239fc6-1fff-4bc5-a580-311b09d6ae25 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:13.394056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 910/1818] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:13.395812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651db215-8eed-42ce-9465-1c2235a27222 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:13.396106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651db215-8eed-42ce-9465-1c2235a27222 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:13.396453 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651db215-8eed-42ce-9465-1c2235a27222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:13.511449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-651db215-8eed-42ce-9465-1c2235a27222 None None] MFA Rules not processed for user `9a5ab9e1e14e48a49293840e74e84d15`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:13.512723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-651db215-8eed-42ce-9465-1c2235a27222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:13.634703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86239fc6-1fff-4bc5-a580-311b09d6ae25 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:13.660930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695626e-f88d-42bb-a9aa-a0ac1ca3ecff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:13.664728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695626e-f88d-42bb-a9aa-a0ac1ca3ecff admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:13.666736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695626e-f88d-42bb-a9aa-a0ac1ca3ecff admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:13.668822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695626e-f88d-42bb-a9aa-a0ac1ca3ecff admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:13.670413 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695626e-f88d-42bb-a9aa-a0ac1ca3ecff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:13.675473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695626e-f88d-42bb-a9aa-a0ac1ca3ecff admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:13.774437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 909/1819] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:13 2022] POST /identity/v3/projects => generated 385 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:13.776393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695626e-f88d-42bb-a9aa-a0ac1ca3ecff admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:13.812645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 911/1820] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:13 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:13.814128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651db215-8eed-42ce-9465-1c2235a27222 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:13.857427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651db215-8eed-42ce-9465-1c2235a27222 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:13.985787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695626e-f88d-42bb-a9aa-a0ac1ca3ecff admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:14.001144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bb500-5979-476d-8189-19b5067d8cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:14.002725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bb500-5979-476d-8189-19b5067d8cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:14.003462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bb500-5979-476d-8189-19b5067d8cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:14.005428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bb500-5979-476d-8189-19b5067d8cd2 admin admin] PATH_INFO: `/v3/projects/220931f9c6904e339265e33cb036de16/users/c34260b9070d45fb9a11a44f8a575ed7/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:14.015931 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:14.048320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bb500-5979-476d-8189-19b5067d8cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=220931f9c6904e339265e33cb036de16, user_id=c34260b9070d45fb9a11a44f8a575ed7, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:14.052087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bb500-5979-476d-8189-19b5067d8cd2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:14.092992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 910/1821] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:13 2022] PUT /identity/v3/projects/220931f9c6904e339265e33cb036de16/users/c34260b9070d45fb9a11a44f8a575ed7/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:14.096528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bb500-5979-476d-8189-19b5067d8cd2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:14.104676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-412bb500-5979-476d-8189-19b5067d8cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:14.108393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301a08b-5301-4fb3-8c92-251d69dfbba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:14.110647 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301a08b-5301-4fb3-8c92-251d69dfbba1 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:14.113176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301a08b-5301-4fb3-8c92-251d69dfbba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:14.114792 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301a08b-5301-4fb3-8c92-251d69dfbba1 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:14.116405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301a08b-5301-4fb3-8c92-251d69dfbba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:14.123949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301a08b-5301-4fb3-8c92-251d69dfbba1 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:14.192164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 912/1822] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:13 2022] POST /identity/v3/users => generated 293 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:14.193808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301a08b-5301-4fb3-8c92-251d69dfbba1 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:14.224056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301a08b-5301-4fb3-8c92-251d69dfbba1 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:14.346401 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:14.347869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bb500-5979-476d-8189-19b5067d8cd2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:14.362780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756c935-3029-4977-8315-065a4455edb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:14.363496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756c935-3029-4977-8315-065a4455edb0 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:14.365856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756c935-3029-4977-8315-065a4455edb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:14.366143 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756c935-3029-4977-8315-065a4455edb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:14.398223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4128a3-ec96-4954-87d9-c39bd18be305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:14.399078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4128a3-ec96-4954-87d9-c39bd18be305 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:14.402103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4128a3-ec96-4954-87d9-c39bd18be305 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:14.404588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4128a3-ec96-4954-87d9-c39bd18be305 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:14.418563 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4128a3-ec96-4954-87d9-c39bd18be305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:14.427637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4128a3-ec96-4954-87d9-c39bd18be305 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:14.470231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 913/1823] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:14.471837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4128a3-ec96-4954-87d9-c39bd18be305 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:14.513120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4128a3-ec96-4954-87d9-c39bd18be305 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:14.533366 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb1cf7-6632-425e-9621-7c9e3f96a78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:14.534444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb1cf7-6632-425e-9621-7c9e3f96a78d admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:14.534929 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb1cf7-6632-425e-9621-7c9e3f96a78d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:14.535400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb1cf7-6632-425e-9621-7c9e3f96a78d admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:14.536761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb1cf7-6632-425e-9621-7c9e3f96a78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:14.540778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb1cf7-6632-425e-9621-7c9e3f96a78d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:14.565940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 914/1824] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:14.567166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb1cf7-6632-425e-9621-7c9e3f96a78d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:14.604857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb1cf7-6632-425e-9621-7c9e3f96a78d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:14.612071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1594f-0902-4e3b-99d4-014739a862a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:14.613800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1594f-0902-4e3b-99d4-014739a862a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:14.614281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1594f-0902-4e3b-99d4-014739a862a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:14.614745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1594f-0902-4e3b-99d4-014739a862a5 admin admin] PATH_INFO: `/v3/projects/220931f9c6904e339265e33cb036de16/users/c34260b9070d45fb9a11a44f8a575ed7/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:14.617528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1594f-0902-4e3b-99d4-014739a862a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=220931f9c6904e339265e33cb036de16, user_id=c34260b9070d45fb9a11a44f8a575ed7, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:14.622586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1594f-0902-4e3b-99d4-014739a862a5 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:14.648543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756c935-3029-4977-8315-065a4455edb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:14.651773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756c935-3029-4977-8315-065a4455edb0 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:14.662414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 915/1825] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:14 2022] PUT /identity/v3/projects/220931f9c6904e339265e33cb036de16/users/c34260b9070d45fb9a11a44f8a575ed7/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:14.674452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1594f-0902-4e3b-99d4-014739a862a5 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:14.679704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 911/1826] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:14 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:14.682656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca87dae-a7d7-4bb3-b7b8-882dae53a902 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:14.683124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca87dae-a7d7-4bb3-b7b8-882dae53a902 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:14.683560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca87dae-a7d7-4bb3-b7b8-882dae53a902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:14.772856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7ca87dae-a7d7-4bb3-b7b8-882dae53a902 None None] MFA Rules not processed for user `c34260b9070d45fb9a11a44f8a575ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:14.773963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7ca87dae-a7d7-4bb3-b7b8-882dae53a902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:14.919912 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1594f-0902-4e3b-99d4-014739a862a5 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:14.944763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9aa75-fff2-4496-a6cb-a580cf99784a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:14.945748 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9aa75-fff2-4496-a6cb-a580cf99784a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:14.946230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9aa75-fff2-4496-a6cb-a580cf99784a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:14.946825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9aa75-fff2-4496-a6cb-a580cf99784a admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:14.948955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9aa75-fff2-4496-a6cb-a580cf99784a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:14.951062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de9aa75-fff2-4496-a6cb-a580cf99784a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:14.952079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 912/1827] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:14.952898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca87dae-a7d7-4bb3-b7b8-882dae53a902 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:14.955967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7ca87dae-a7d7-4bb3-b7b8-882dae53a902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:14.995473 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 916/1828] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:14.999014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9aa75-fff2-4496-a6cb-a580cf99784a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:15.004721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:15.100992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca87dae-a7d7-4bb3-b7b8-882dae53a902 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:15.111662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02494866-c579-449b-b3a3-e5bd789a7b52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:15.112443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02494866-c579-449b-b3a3-e5bd789a7b52 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:15.114569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02494866-c579-449b-b3a3-e5bd789a7b52 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:15.114935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02494866-c579-449b-b3a3-e5bd789a7b52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:15.122003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-02494866-c579-449b-b3a3-e5bd789a7b52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:15.144432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9aa75-fff2-4496-a6cb-a580cf99784a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:15.156723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baf4a3-26f2-477e-806a-a46cc692d1d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:15.157489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baf4a3-26f2-477e-806a-a46cc692d1d7 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:15.157931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baf4a3-26f2-477e-806a-a46cc692d1d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:15.158301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baf4a3-26f2-477e-806a-a46cc692d1d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:15.184566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76baf4a3-26f2-477e-806a-a46cc692d1d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:15.187923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76baf4a3-26f2-477e-806a-a46cc692d1d7 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:15.227399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 917/1829] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:15.230123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baf4a3-26f2-477e-806a-a46cc692d1d7 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:15.308476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baf4a3-26f2-477e-806a-a46cc692d1d7 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:15.314022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02494866-c579-449b-b3a3-e5bd789a7b52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:15.317146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02494866-c579-449b-b3a3-e5bd789a7b52 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:15.328489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0635f-7d51-406e-b3c3-48c856349bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:15.331742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0635f-7d51-406e-b3c3-48c856349bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:15.332905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0635f-7d51-406e-b3c3-48c856349bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:15.333433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0635f-7d51-406e-b3c3-48c856349bd0 admin admin] PATH_INFO: `/v3/projects/a13abdc4d0524656984430c3aea7e95a/users/30ecd48609d74d7a823222f06c4ad7d5/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:15.360621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 913/1830] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:15.369610 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0635f-7d51-406e-b3c3-48c856349bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a13abdc4d0524656984430c3aea7e95a, user_id=30ecd48609d74d7a823222f06c4ad7d5, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:15.373922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0635f-7d51-406e-b3c3-48c856349bd0 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:15.404423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 918/1831] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:15 2022] PUT /identity/v3/projects/a13abdc4d0524656984430c3aea7e95a/users/30ecd48609d74d7a823222f06c4ad7d5/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:15.417006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced840d7-c328-40d0-80b4-3baa9a123992 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:15.417538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced840d7-c328-40d0-80b4-3baa9a123992 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:15.417945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced840d7-c328-40d0-80b4-3baa9a123992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:15.519052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ced840d7-c328-40d0-80b4-3baa9a123992 None None] MFA Rules not processed for user `30ecd48609d74d7a823222f06c4ad7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:15.520538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ced840d7-c328-40d0-80b4-3baa9a123992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:15.707467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efd49b-352e-4017-87aa-d315d6614028 None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:15.707976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efd49b-352e-4017-87aa-d315d6614028 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:15.708484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efd49b-352e-4017-87aa-d315d6614028 None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:15.710616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 919/1832] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:53:15.732210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 914/1833] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:15.737171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efd49b-352e-4017-87aa-d315d6614028 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:15.772947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced840d7-c328-40d0-80b4-3baa9a123992 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:15.775457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ced840d7-c328-40d0-80b4-3baa9a123992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:15.835444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:15.909416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efd49b-352e-4017-87aa-d315d6614028 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:15.922426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac5709-fbe0-4ede-86ab-1343153df3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:15.924279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac5709-fbe0-4ede-86ab-1343153df3db admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:15.927113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac5709-fbe0-4ede-86ab-1343153df3db admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:15.927520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac5709-fbe0-4ede-86ab-1343153df3db admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:15.928409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac5709-fbe0-4ede-86ab-1343153df3db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:15.930490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac5709-fbe0-4ede-86ab-1343153df3db admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:15.957001 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced840d7-c328-40d0-80b4-3baa9a123992 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:15.968791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:15.972701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:15.974840 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:15.975456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] PATH_INFO: `/v3/users/5543933aa25b4dccb845ca336bf45dea` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:15.980897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5543933aa25b4dccb845ca336bf45dea)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:15.985666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:16.018299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:16.025512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 920/1834] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:15 2022] POST /identity/v3/projects => generated 385 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:16.025512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac5709-fbe0-4ede-86ab-1343153df3db admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:16.142364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5543933aa25b4dccb845ca336bf45dea'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:16.201802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5543933aa25b4dccb845ca336bf45dea'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:16.219703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5543933aa25b4dccb845ca336bf45dea'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:16.221278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac5709-fbe0-4ede-86ab-1343153df3db admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:16.238862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce22e65-a18c-4e7f-8745-c9903fc70bec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:16.241381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce22e65-a18c-4e7f-8745-c9903fc70bec service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:16.242948 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce22e65-a18c-4e7f-8745-c9903fc70bec service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:16.243834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce22e65-a18c-4e7f-8745-c9903fc70bec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:16.260491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5543933aa25b4dccb845ca336bf45dea'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:16.262570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c306cbae-edcb-42a5-8da3-71232ed0a7a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5543933aa25b4dccb845ca336bf45dea'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:16.278371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 915/1835] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:15 2022] DELETE /identity/v3/users/5543933aa25b4dccb845ca336bf45dea => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:16.279717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a23f9e-349b-4fc6-a0f5-a233805970d0 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:16.280022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a23f9e-349b-4fc6-a0f5-a233805970d0 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:16.280277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a23f9e-349b-4fc6-a0f5-a233805970d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:16.331090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-74a23f9e-349b-4fc6-a0f5-a233805970d0 None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:16.332656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-74a23f9e-349b-4fc6-a0f5-a233805970d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:16.447179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce22e65-a18c-4e7f-8745-c9903fc70bec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:16.451509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce22e65-a18c-4e7f-8745-c9903fc70bec service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:16.469431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 921/1836] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:16.471833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce22e65-a18c-4e7f-8745-c9903fc70bec service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:16.592380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 916/1837] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:53:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:16.594577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce22e65-a18c-4e7f-8745-c9903fc70bec service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:16.595559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd660b09-222d-40ab-85d6-d2b8c611fd5f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:16.596196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd660b09-222d-40ab-85d6-d2b8c611fd5f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:16.597045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd660b09-222d-40ab-85d6-d2b8c611fd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:16.628796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:16.638619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-dd660b09-222d-40ab-85d6-d2b8c611fd5f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:16.643170 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-dd660b09-222d-40ab-85d6-d2b8c611fd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:16.707461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f017ff-5d00-4b72-8a2c-9f51767b951c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:16.708670 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f017ff-5d00-4b72-8a2c-9f51767b951c admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:16.709178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f017ff-5d00-4b72-8a2c-9f51767b951c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:16.709743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f017ff-5d00-4b72-8a2c-9f51767b951c admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:16.710986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f017ff-5d00-4b72-8a2c-9f51767b951c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:16.715161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f017ff-5d00-4b72-8a2c-9f51767b951c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:16.779760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 922/1838] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:16 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:16.783521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f017ff-5d00-4b72-8a2c-9f51767b951c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:16.785919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-54f017ff-5d00-4b72-8a2c-9f51767b951c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:16.816030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 917/1839] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:16.821376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f017ff-5d00-4b72-8a2c-9f51767b951c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:16.829434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efdf50-1fb0-47d7-9245-ea43f8af68ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:16.831585 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efdf50-1fb0-47d7-9245-ea43f8af68ff admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:16.832634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efdf50-1fb0-47d7-9245-ea43f8af68ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:16.833286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efdf50-1fb0-47d7-9245-ea43f8af68ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:16.834481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efdf50-1fb0-47d7-9245-ea43f8af68ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:16.837612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efdf50-1fb0-47d7-9245-ea43f8af68ff admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:16.892677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 923/1840] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:16.893532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd660b09-222d-40ab-85d6-d2b8c611fd5f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:16.910108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd660b09-222d-40ab-85d6-d2b8c611fd5f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:16.922728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11dec6-7f84-491f-8ac8-b128d1a0271c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:16.923968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11dec6-7f84-491f-8ac8-b128d1a0271c admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:16.923968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11dec6-7f84-491f-8ac8-b128d1a0271c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:16.923968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11dec6-7f84-491f-8ac8-b128d1a0271c admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:16.924919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11dec6-7f84-491f-8ac8-b128d1a0271c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:16.926665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11dec6-7f84-491f-8ac8-b128d1a0271c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:16.960594 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 918/1841] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:16.964974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efdf50-1fb0-47d7-9245-ea43f8af68ff admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:16.984236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efdf50-1fb0-47d7-9245-ea43f8af68ff admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:17.004116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11dec6-7f84-491f-8ac8-b128d1a0271c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:17.005985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2d11dec6-7f84-491f-8ac8-b128d1a0271c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:17.015622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:17.021602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbac96c-1ca5-40b2-a83e-1d0a3afce887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:17.021602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbac96c-1ca5-40b2-a83e-1d0a3afce887 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:17.021989 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbac96c-1ca5-40b2-a83e-1d0a3afce887 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:17.022961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbac96c-1ca5-40b2-a83e-1d0a3afce887 admin admin] PATH_INFO: `/v3/projects/87bc86ed11294134b70f5135ad7418e2/users/7f67e5d9abad4fc28bc4ac35681604be/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:17.070469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbac96c-1ca5-40b2-a83e-1d0a3afce887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87bc86ed11294134b70f5135ad7418e2, user_id=7f67e5d9abad4fc28bc4ac35681604be, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:17.075536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbac96c-1ca5-40b2-a83e-1d0a3afce887 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:17.115391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 924/1842] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:16 2022] PUT /identity/v3/projects/87bc86ed11294134b70f5135ad7418e2/users/7f67e5d9abad4fc28bc4ac35681604be/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:17.127665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340ad18-bc83-49bb-b7e2-c388adccc7dc None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:17.128152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340ad18-bc83-49bb-b7e2-c388adccc7dc None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:17.128696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340ad18-bc83-49bb-b7e2-c388adccc7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:17.213078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11dec6-7f84-491f-8ac8-b128d1a0271c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:17.235069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a340ad18-bc83-49bb-b7e2-c388adccc7dc None None] MFA Rules not processed for user `7f67e5d9abad4fc28bc4ac35681604be`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:17.245358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a340ad18-bc83-49bb-b7e2-c388adccc7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:17.377694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec910e3-5630-4f7f-9538-947496c8ea01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:17.379811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec910e3-5630-4f7f-9538-947496c8ea01 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:17.380572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec910e3-5630-4f7f-9538-947496c8ea01 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:17.381043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec910e3-5630-4f7f-9538-947496c8ea01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:17.537661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 925/1843] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:17.539142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340ad18-bc83-49bb-b7e2-c388adccc7dc None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:17.630177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec910e3-5630-4f7f-9538-947496c8ea01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:17.638172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec910e3-5630-4f7f-9538-947496c8ea01 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:17.653528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 919/1844] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:17.725920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340ad18-bc83-49bb-b7e2-c388adccc7dc None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:17.746842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774331e-e631-453f-9791-6cca97efba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:17.747495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774331e-e631-453f-9791-6cca97efba6c admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:17.748189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774331e-e631-453f-9791-6cca97efba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:17.748189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774331e-e631-453f-9791-6cca97efba6c admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:17.748737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774331e-e631-453f-9791-6cca97efba6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:17.750998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774331e-e631-453f-9791-6cca97efba6c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:17.777985 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:17.819141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 926/1845] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:17 2022] POST /identity/v3/projects => generated 383 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:53:17.820457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec910e3-5630-4f7f-9538-947496c8ea01 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:17.843804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec910e3-5630-4f7f-9538-947496c8ea01 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:18.039802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37477292-4881-431a-97d3-fcc36a395602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:18.041008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37477292-4881-431a-97d3-fcc36a395602 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:18.041198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37477292-4881-431a-97d3-fcc36a395602 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:18.041472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37477292-4881-431a-97d3-fcc36a395602 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:18.042402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37477292-4881-431a-97d3-fcc36a395602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:18.053514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37477292-4881-431a-97d3-fcc36a395602 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:18.106565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:18.120617 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 920/1846] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:17 2022] POST /identity/v3/users => generated 298 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:18.122201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774331e-e631-453f-9791-6cca97efba6c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:18.137509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774331e-e631-453f-9791-6cca97efba6c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:18.148633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37477292-4881-431a-97d3-fcc36a395602 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:18.150656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-37477292-4881-431a-97d3-fcc36a395602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:18.160137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d61d26-3fe6-4cfd-b3b7-71b32c87bcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:18.160823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61d26-3fe6-4cfd-b3b7-71b32c87bcc4 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:18.161106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61d26-3fe6-4cfd-b3b7-71b32c87bcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:18.164655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61d26-3fe6-4cfd-b3b7-71b32c87bcc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:18.166130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d61d26-3fe6-4cfd-b3b7-71b32c87bcc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:18.167838 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d61d26-3fe6-4cfd-b3b7-71b32c87bcc4 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:18.222476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 927/1847] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:18.228239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d61d26-3fe6-4cfd-b3b7-71b32c87bcc4 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:18.239784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d61d26-3fe6-4cfd-b3b7-71b32c87bcc4 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:18.263891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa42702-c75e-4b71-8d3a-fb06f5153726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:18.264511 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa42702-c75e-4b71-8d3a-fb06f5153726 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:18.264846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa42702-c75e-4b71-8d3a-fb06f5153726 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:18.265126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa42702-c75e-4b71-8d3a-fb06f5153726 admin admin] PATH_INFO: `/v3/projects/9df8a071f321415ab8a8e66e354f64b6/users/2768bd566ed04158b94cde86136a6c09/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:18.331873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa42702-c75e-4b71-8d3a-fb06f5153726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df8a071f321415ab8a8e66e354f64b6, user_id=2768bd566ed04158b94cde86136a6c09, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:18.354186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa42702-c75e-4b71-8d3a-fb06f5153726 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:18.415798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 928/1848] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:18 2022] PUT /identity/v3/projects/9df8a071f321415ab8a8e66e354f64b6/users/2768bd566ed04158b94cde86136a6c09/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:18.416697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37477292-4881-431a-97d3-fcc36a395602 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:18.422805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa42702-c75e-4b71-8d3a-fb06f5153726 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:18.560605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674b8d9-ab81-4796-aaf3-ff184842850a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:18.561112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674b8d9-ab81-4796-aaf3-ff184842850a service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:18.561560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674b8d9-ab81-4796-aaf3-ff184842850a service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:18.563375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674b8d9-ab81-4796-aaf3-ff184842850a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:18.574945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674b8d9-ab81-4796-aaf3-ff184842850a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:18.576686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674b8d9-ab81-4796-aaf3-ff184842850a service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:18.593049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 921/1849] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:53:18.595510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264d219-827e-4d43-ac9b-c940a155d9e1 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:18.595848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264d219-827e-4d43-ac9b-c940a155d9e1 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:18.596300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264d219-827e-4d43-ac9b-c940a155d9e1 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:18.597981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 922/1850] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:53:18.699310 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa42702-c75e-4b71-8d3a-fb06f5153726 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:18.713622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f857b-bd3c-48a8-a746-1ae51716f078 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:18.714178 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f857b-bd3c-48a8-a746-1ae51716f078 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:18.714432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f857b-bd3c-48a8-a746-1ae51716f078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:18.725631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc7937-a018-4e24-8612-885951c2e6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:18.730893 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc7937-a018-4e24-8612-885951c2e6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:18.731176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc7937-a018-4e24-8612-885951c2e6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:18.731453 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc7937-a018-4e24-8612-885951c2e6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:18.740486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc7937-a018-4e24-8612-885951c2e6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:18.752427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc7937-a018-4e24-8612-885951c2e6d2 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:18.770876 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-211f857b-bd3c-48a8-a746-1ae51716f078 None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:18.772113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-211f857b-bd3c-48a8-a746-1ae51716f078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:18.793898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 929/1851] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:18.799633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc7937-a018-4e24-8612-885951c2e6d2 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:18.805751 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 923/1852] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:53:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:18.827273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc7937-a018-4e24-8612-885951c2e6d2 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:18.841143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d104b-0b7f-449d-b249-940f96a4ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:18.844791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d104b-0b7f-449d-b249-940f96a4ee7f admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:18.845081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d104b-0b7f-449d-b249-940f96a4ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:18.846238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d104b-0b7f-449d-b249-940f96a4ee7f admin admin] PATH_INFO: `/v3/projects/9df8a071f321415ab8a8e66e354f64b6/users/2768bd566ed04158b94cde86136a6c09/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:18.856084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d104b-0b7f-449d-b249-940f96a4ee7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df8a071f321415ab8a8e66e354f64b6, user_id=2768bd566ed04158b94cde86136a6c09, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:18.880036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d104b-0b7f-449d-b249-940f96a4ee7f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:18.918402 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 930/1853] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:18 2022] PUT /identity/v3/projects/9df8a071f321415ab8a8e66e354f64b6/users/2768bd566ed04158b94cde86136a6c09/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:18.929974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d13d88-3d3e-4c54-bc4d-8e8bb6a8fcf0 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:18.930402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d13d88-3d3e-4c54-bc4d-8e8bb6a8fcf0 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:18.930526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d13d88-3d3e-4c54-bc4d-8e8bb6a8fcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:19.005607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-08d13d88-3d3e-4c54-bc4d-8e8bb6a8fcf0 None None] MFA Rules not processed for user `2768bd566ed04158b94cde86136a6c09`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:19.006977 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-08d13d88-3d3e-4c54-bc4d-8e8bb6a8fcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:19.128875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 924/1854] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:19.155869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d104b-0b7f-449d-b249-940f96a4ee7f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:19.157870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0e1d104b-0b7f-449d-b249-940f96a4ee7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:19.276212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d104b-0b7f-449d-b249-940f96a4ee7f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:19.281303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8b3bc-6b40-4d93-9653-2fc48d9d26b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:19.282366 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b3bc-6b40-4d93-9653-2fc48d9d26b6 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:19.282648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b3bc-6b40-4d93-9653-2fc48d9d26b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:19.282923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8b3bc-6b40-4d93-9653-2fc48d9d26b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:19.284935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-15f8b3bc-6b40-4d93-9653-2fc48d9d26b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:19.375296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d13d88-3d3e-4c54-bc4d-8e8bb6a8fcf0 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:19.380445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:19.438481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:19.476203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f8b3bc-6b40-4d93-9653-2fc48d9d26b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:19.480725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f8b3bc-6b40-4d93-9653-2fc48d9d26b6 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:19.526608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 931/1855] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:19.569006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d13d88-3d3e-4c54-bc4d-8e8bb6a8fcf0 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:19.579210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdb5ab-5db1-4052-b10b-6a844c90c2ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:19.580183 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdb5ab-5db1-4052-b10b-6a844c90c2ae service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:19.580655 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdb5ab-5db1-4052-b10b-6a844c90c2ae service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:19.581113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdb5ab-5db1-4052-b10b-6a844c90c2ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:19.603628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdb5ab-5db1-4052-b10b-6a844c90c2ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:19.605626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdb5ab-5db1-4052-b10b-6a844c90c2ae service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:19.630869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 925/1856] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:20.126612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8b3bc-6b40-4d93-9653-2fc48d9d26b6 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:20.128088 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-15f8b3bc-6b40-4d93-9653-2fc48d9d26b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:20.178968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdb5ab-5db1-4052-b10b-6a844c90c2ae service nova] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:20.180420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6bbdb5ab-5db1-4052-b10b-6a844c90c2ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:20.224982 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8b3bc-6b40-4d93-9653-2fc48d9d26b6 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:20.231638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:20.232799 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:20.233287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:20.233667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] PATH_INFO: `/v3/users/13f50e4374204f36a24268f302304625` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:20.235087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f50e4374204f36a24268f302304625)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:20.237212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:20.347977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f50e4374204f36a24268f302304625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.348669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdb5ab-5db1-4052-b10b-6a844c90c2ae service nova] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:20.359149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:20.360168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:20.360679 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:20.361089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] PATH_INFO: `/v3/projects/2bc78a09cde84f10b2c33fa7a2332f92` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:20.362610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bc78a09cde84f10b2c33fa7a2332f92)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:20.365047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:20.366983 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f50e4374204f36a24268f302304625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.368745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f50e4374204f36a24268f302304625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.389933 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f50e4374204f36a24268f302304625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.391160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8a6842c5-abce-4d66-8a8f-25371702be9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f50e4374204f36a24268f302304625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.412448 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 932/1857] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:20 2022] DELETE /identity/v3/users/13f50e4374204f36a24268f302304625 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:20.418674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b385ca7-23bc-48a6-93ff-fc05dddf7ac0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:20.418895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b385ca7-23bc-48a6-93ff-fc05dddf7ac0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:20.419239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b385ca7-23bc-48a6-93ff-fc05dddf7ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:20.447748 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9b385ca7-23bc-48a6-93ff-fc05dddf7ac0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:20.449887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9b385ca7-23bc-48a6-93ff-fc05dddf7ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:20.460236 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bc78a09cde84f10b2c33fa7a2332f92', 'request_id': 'req-0581a076-ecfa-420d-8523-1ca3ab26d710'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.475955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bc78a09cde84f10b2c33fa7a2332f92', 'request_id': 'req-0581a076-ecfa-420d-8523-1ca3ab26d710'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.486597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bc78a09cde84f10b2c33fa7a2332f92', 'request_id': 'req-0581a076-ecfa-420d-8523-1ca3ab26d710'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.502675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] The token cache is being invalidate because project 2bc78a09cde84f10b2c33fa7a2332f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:20.502978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.504013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.506677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 926/1858] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:20 2022] DELETE /identity/v3/projects/2bc78a09cde84f10b2c33fa7a2332f92 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:20.510994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:20.514262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:20.547968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:20.566634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 933/1859] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:20.579315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b385ca7-23bc-48a6-93ff-fc05dddf7ac0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:20.581053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9b385ca7-23bc-48a6-93ff-fc05dddf7ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:20.601016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:20.666577 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581a076-ecfa-420d-8523-1ca3ab26d710 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:20.672462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:20.673403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:20.673721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:20.673905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b385ca7-23bc-48a6-93ff-fc05dddf7ac0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:20.674006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] PATH_INFO: `/v3/users/c9bedf982994494e87cee97d97e96f83` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:20.675223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9bedf982994494e87cee97d97e96f83)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:20.677916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:20.678043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4787d7f-e261-4b3c-a711-ea5f731c72b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:20.678729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4787d7f-e261-4b3c-a711-ea5f731c72b6 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:20.679080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4787d7f-e261-4b3c-a711-ea5f731c72b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:20.679428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4787d7f-e261-4b3c-a711-ea5f731c72b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:20.692892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4787d7f-e261-4b3c-a711-ea5f731c72b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:20.694170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4787d7f-e261-4b3c-a711-ea5f731c72b6 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:20.712608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 934/1860] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:20.713781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4787d7f-e261-4b3c-a711-ea5f731c72b6 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:20.716117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c4787d7f-e261-4b3c-a711-ea5f731c72b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:20.740840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4787d7f-e261-4b3c-a711-ea5f731c72b6 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:20.746476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:20.747468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:20.747902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:20.748008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9bedf982994494e87cee97d97e96f83'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.748351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] PATH_INFO: `/v3/users/ed9df1a4183446af94d57f655443b77c` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:20.749995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed9df1a4183446af94d57f655443b77c)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:20.752045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:20.767938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9bedf982994494e87cee97d97e96f83'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.778949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9bedf982994494e87cee97d97e96f83'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.789049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9bedf982994494e87cee97d97e96f83'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.790410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-36327ab0-83ef-45e9-acbf-442a8aa4b2bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9bedf982994494e87cee97d97e96f83'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.803550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 927/1861] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:20 2022] DELETE /identity/v3/users/c9bedf982994494e87cee97d97e96f83 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:20.810850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed9df1a4183446af94d57f655443b77c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.818923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed9df1a4183446af94d57f655443b77c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.819627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed9df1a4183446af94d57f655443b77c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.835219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed9df1a4183446af94d57f655443b77c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.836192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed9df1a4183446af94d57f655443b77c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:20.858144 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 935/1862] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:20 2022] DELETE /identity/v3/users/ed9df1a4183446af94d57f655443b77c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:20.862723 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f00f7-de86-45bd-9b0b-fbfb266d2d8d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:20.863006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f00f7-de86-45bd-9b0b-fbfb266d2d8d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:20.863278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f00f7-de86-45bd-9b0b-fbfb266d2d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:20.893501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a09f00f7-de86-45bd-9b0b-fbfb266d2d8d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:20.895425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a09f00f7-de86-45bd-9b0b-fbfb266d2d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:20.985670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 928/1863] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:21.034994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:21.036826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:21.103231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535b8bc-d194-4965-a1b7-1eedf3b4fa84 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:21.107624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013422aa-6376-460e-afc7-70b2def2701f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:21.108239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013422aa-6376-460e-afc7-70b2def2701f service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:21.108524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013422aa-6376-460e-afc7-70b2def2701f service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:21.108954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013422aa-6376-460e-afc7-70b2def2701f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:21.123966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013422aa-6376-460e-afc7-70b2def2701f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:21.125544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013422aa-6376-460e-afc7-70b2def2701f service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:21.138922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 936/1864] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:21.555456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f00f7-de86-45bd-9b0b-fbfb266d2d8d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:21.559698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a09f00f7-de86-45bd-9b0b-fbfb266d2d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:21.614728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:21.641550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f00f7-de86-45bd-9b0b-fbfb266d2d8d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:21.645128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205493b4-7651-4cc8-b8a6-6e1f16a996e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:21.645955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205493b4-7651-4cc8-b8a6-6e1f16a996e7 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:21.646817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205493b4-7651-4cc8-b8a6-6e1f16a996e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:21.646817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205493b4-7651-4cc8-b8a6-6e1f16a996e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:21.649132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-205493b4-7651-4cc8-b8a6-6e1f16a996e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:21.719978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205493b4-7651-4cc8-b8a6-6e1f16a996e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:21.721451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205493b4-7651-4cc8-b8a6-6e1f16a996e7 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:21.741862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 929/1865] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:53:22.023080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013422aa-6376-460e-afc7-70b2def2701f service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:22.026586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205493b4-7651-4cc8-b8a6-6e1f16a996e7 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:22.029080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-013422aa-6376-460e-afc7-70b2def2701f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:22.031678 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-205493b4-7651-4cc8-b8a6-6e1f16a996e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:22.080724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013422aa-6376-460e-afc7-70b2def2701f service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:22.086216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:22.088509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:22.089548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:22.089889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] PATH_INFO: `/v3/projects/b70f786b912248f7ab167a39bf7c1f70` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:22.092347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b70f786b912248f7ab167a39bf7c1f70)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:22.094819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205493b4-7651-4cc8-b8a6-6e1f16a996e7 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:22.095672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:22.106398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:22.107289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:22.107571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:22.107849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] PATH_INFO: `/v3/projects/22623461dc0245a5afbda0f49e80f161` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:22.110782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=22623461dc0245a5afbda0f49e80f161)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:22.115974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:22.152009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:22.215102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b70f786b912248f7ab167a39bf7c1f70', 'request_id': 'req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.242074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22623461dc0245a5afbda0f49e80f161', 'request_id': 'req-885b6709-7c2f-4d3a-b064-c77127177c9c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.244212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b70f786b912248f7ab167a39bf7c1f70', 'request_id': 'req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.255177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b70f786b912248f7ab167a39bf7c1f70', 'request_id': 'req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.264458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22623461dc0245a5afbda0f49e80f161', 'request_id': 'req-885b6709-7c2f-4d3a-b064-c77127177c9c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.280540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22623461dc0245a5afbda0f49e80f161', 'request_id': 'req-885b6709-7c2f-4d3a-b064-c77127177c9c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.288974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] The token cache is being invalidate because project b70f786b912248f7ab167a39bf7c1f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:22.289369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.292035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.298715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 937/1866] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:22 2022] DELETE /identity/v3/projects/b70f786b912248f7ab167a39bf7c1f70 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:22.299591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:22.315214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:22.320288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] The token cache is being invalidate because project 22623461dc0245a5afbda0f49e80f161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:22.320650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.322744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-885b6709-7c2f-4d3a-b064-c77127177c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.324884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 930/1867] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:22 2022] DELETE /identity/v3/projects/22623461dc0245a5afbda0f49e80f161 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:22.334406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70bd1c-a944-4382-acb9-6a859bff47a0 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:22.335068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70bd1c-a944-4382-acb9-6a859bff47a0 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:22.335429 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70bd1c-a944-4382-acb9-6a859bff47a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:22.374966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-dd70bd1c-a944-4382-acb9-6a859bff47a0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:22.376551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-dd70bd1c-a944-4382-acb9-6a859bff47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:22.492218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eea345-18c0-4a7c-97f5-cc4cf2bc0b6b admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:22.562303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 931/1868] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:22.564107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70bd1c-a944-4382-acb9-6a859bff47a0 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:22.568986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-dd70bd1c-a944-4382-acb9-6a859bff47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:22.598340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70bd1c-a944-4382-acb9-6a859bff47a0 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:22.609027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:22.610251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:22.610682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:22.611133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] PATH_INFO: `/v3/projects/f97e11323fba4bd6ba4f55b97a3b14e4` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:22.615092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97e11323fba4bd6ba4f55b97a3b14e4)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:22.617954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:22.636603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:22.638201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:22.638759 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:22.638759 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] PATH_INFO: `/v3/users/c34260b9070d45fb9a11a44f8a575ed7` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:22.640268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34260b9070d45fb9a11a44f8a575ed7)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:22.641100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:22.644039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:22.705559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97e11323fba4bd6ba4f55b97a3b14e4', 'request_id': 'req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.716921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97e11323fba4bd6ba4f55b97a3b14e4', 'request_id': 'req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.723432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97e11323fba4bd6ba4f55b97a3b14e4', 'request_id': 'req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.729143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34260b9070d45fb9a11a44f8a575ed7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.737352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] The token cache is being invalidate because project f97e11323fba4bd6ba4f55b97a3b14e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:22.737607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.738376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-af267136-5cc9-47f1-9ee9-81a8ae0bdf49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.741158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34260b9070d45fb9a11a44f8a575ed7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.744690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 932/1869] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:22 2022] DELETE /identity/v3/projects/f97e11323fba4bd6ba4f55b97a3b14e4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:22.744690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94458b-1f1e-4e04-9e73-8f21f1dae626 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:22.744690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94458b-1f1e-4e04-9e73-8f21f1dae626 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:22.744690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94458b-1f1e-4e04-9e73-8f21f1dae626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:22.745924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34260b9070d45fb9a11a44f8a575ed7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.752898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:22.763097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34260b9070d45fb9a11a44f8a575ed7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.764521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34260b9070d45fb9a11a44f8a575ed7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:22.773885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-2a94458b-1f1e-4e04-9e73-8f21f1dae626 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:22.778214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2a94458b-1f1e-4e04-9e73-8f21f1dae626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:22.784743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 938/1870] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:22 2022] DELETE /identity/v3/users/c34260b9070d45fb9a11a44f8a575ed7 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:22.785831 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:22.787638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:22.897969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503948b-8f1c-4a9a-a28d-0342b7815483 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:22.905822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:22.907638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:22.908090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:22.908416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] PATH_INFO: `/v3/users/9a5ab9e1e14e48a49293840e74e84d15` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:22.910164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a5ab9e1e14e48a49293840e74e84d15)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:22.912578 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:22.944394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 933/1871] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:22.947969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94458b-1f1e-4e04-9e73-8f21f1dae626 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:22.952759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2a94458b-1f1e-4e04-9e73-8f21f1dae626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:22.996755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a5ab9e1e14e48a49293840e74e84d15'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:23.014793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a5ab9e1e14e48a49293840e74e84d15'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:23.016826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a5ab9e1e14e48a49293840e74e84d15'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:23.050087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94458b-1f1e-4e04-9e73-8f21f1dae626 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:23.075126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a5ab9e1e14e48a49293840e74e84d15'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:23.075902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a5ab9e1e14e48a49293840e74e84d15'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:23.109338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 939/1872] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:22 2022] DELETE /identity/v3/users/9a5ab9e1e14e48a49293840e74e84d15 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:23.112560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:23.123440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:23.169497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1398e-3a7c-4b3c-8a0d-ae22020e4a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:23.170354 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1398e-3a7c-4b3c-8a0d-ae22020e4a33 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:23.172346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1398e-3a7c-4b3c-8a0d-ae22020e4a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:23.172734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1398e-3a7c-4b3c-8a0d-ae22020e4a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:23.178351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-63e1398e-3a7c-4b3c-8a0d-ae22020e4a33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:23.206757 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:23.222201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:23.227802 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fe79b-677b-4b2c-9fa5-ad336e017436 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:23.235909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a833e-51c0-4934-aebb-afb9dee2daec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:23.236795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a833e-51c0-4934-aebb-afb9dee2daec admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:23.237080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a833e-51c0-4934-aebb-afb9dee2daec admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:23.237485 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a833e-51c0-4934-aebb-afb9dee2daec admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:23.238580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a833e-51c0-4934-aebb-afb9dee2daec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:23.238935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a833e-51c0-4934-aebb-afb9dee2daec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:23.241366 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a833e-51c0-4934-aebb-afb9dee2daec admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:23.266113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 940/1873] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:23.272116 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a833e-51c0-4934-aebb-afb9dee2daec admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:23.314822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a833e-51c0-4934-aebb-afb9dee2daec admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:23.322896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cec15-b132-4958-b037-b2c3f0725387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:23.323443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cec15-b132-4958-b037-b2c3f0725387 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:23.323724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cec15-b132-4958-b037-b2c3f0725387 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:23.324317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cec15-b132-4958-b037-b2c3f0725387 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:23.333867 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cec15-b132-4958-b037-b2c3f0725387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:23.333867 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cec15-b132-4958-b037-b2c3f0725387 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:23.379662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1398e-3a7c-4b3c-8a0d-ae22020e4a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:23.384291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1398e-3a7c-4b3c-8a0d-ae22020e4a33 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:23.412080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 934/1874] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:23.467401 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 941/1875] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:23 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:53:23.469229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1398e-3a7c-4b3c-8a0d-ae22020e4a33 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:23.486786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1398e-3a7c-4b3c-8a0d-ae22020e4a33 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:23.661470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d1c07-6ead-4e8d-89ea-c968676cdcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:23.662195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d1c07-6ead-4e8d-89ea-c968676cdcc0 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:23.662473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d1c07-6ead-4e8d-89ea-c968676cdcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:23.662742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d1c07-6ead-4e8d-89ea-c968676cdcc0 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:23.663436 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d1c07-6ead-4e8d-89ea-c968676cdcc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:23.666130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d1c07-6ead-4e8d-89ea-c968676cdcc0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:23.692776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cec15-b132-4958-b037-b2c3f0725387 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:23.694958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ce6cec15-b132-4958-b037-b2c3f0725387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:23.711076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 935/1876] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:23 2022] POST /identity/v3/users => generated 291 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:23.713039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d1c07-6ead-4e8d-89ea-c968676cdcc0 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:23.720701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cec15-b132-4958-b037-b2c3f0725387 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:23.729963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:23.731499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:23.732005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:23.732487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] PATH_INFO: `/v3/projects/220931f9c6904e339265e33cb036de16` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:23.733796 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d1c07-6ead-4e8d-89ea-c968676cdcc0 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:23.734530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=220931f9c6904e339265e33cb036de16)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:23.737050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:23.749065 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e2b10-ae38-485a-bf7b-0273638a4cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:23.749791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e2b10-ae38-485a-bf7b-0273638a4cad admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:23.750045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e2b10-ae38-485a-bf7b-0273638a4cad admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:23.750316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e2b10-ae38-485a-bf7b-0273638a4cad admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:23.751043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e2b10-ae38-485a-bf7b-0273638a4cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:23.752656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e2b10-ae38-485a-bf7b-0273638a4cad admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:23.771394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 936/1877] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:23.778120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e2b10-ae38-485a-bf7b-0273638a4cad admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:23.792587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e2b10-ae38-485a-bf7b-0273638a4cad admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:23.824851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0b404-e4f2-4bd1-8b6c-392dc0f80ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:23.829635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0b404-e4f2-4bd1-8b6c-392dc0f80ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:23.831703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0b404-e4f2-4bd1-8b6c-392dc0f80ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:23.832634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0b404-e4f2-4bd1-8b6c-392dc0f80ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:23.835040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0b404-e4f2-4bd1-8b6c-392dc0f80ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:23.850489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0b404-e4f2-4bd1-8b6c-392dc0f80ac8 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:23.880506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '220931f9c6904e339265e33cb036de16', 'request_id': 'req-ca37b785-3b06-40d9-978f-f9584e0799fc'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:23.901544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '220931f9c6904e339265e33cb036de16', 'request_id': 'req-ca37b785-3b06-40d9-978f-f9584e0799fc'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:23.908077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 937/1878] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:23.910327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '220931f9c6904e339265e33cb036de16', 'request_id': 'req-ca37b785-3b06-40d9-978f-f9584e0799fc'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:23.911157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0b404-e4f2-4bd1-8b6c-392dc0f80ac8 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:23.926302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0b404-e4f2-4bd1-8b6c-392dc0f80ac8 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:23.932778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] The token cache is being invalidate because project 220931f9c6904e339265e33cb036de16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:23.933252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:23.934603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ca37b785-3b06-40d9-978f-f9584e0799fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:23.938467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 942/1879] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:23 2022] DELETE /identity/v3/projects/220931f9c6904e339265e33cb036de16 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:23.943637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bd834-32bd-43b8-a54b-ee640cf3d3e6 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:23.944349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bd834-32bd-43b8-a54b-ee640cf3d3e6 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:23.944942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bd834-32bd-43b8-a54b-ee640cf3d3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:23.985116 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-388bd834-32bd-43b8-a54b-ee640cf3d3e6 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:23.989121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-388bd834-32bd-43b8-a54b-ee640cf3d3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:24.056540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff955c2a-1a20-4eb3-93b2-7291e7ddcbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.058003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955c2a-1a20-4eb3-93b2-7291e7ddcbed admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.058446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955c2a-1a20-4eb3-93b2-7291e7ddcbed admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.058889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff955c2a-1a20-4eb3-93b2-7291e7ddcbed admin admin] PATH_INFO: `/v3/projects/6df40e616f5f471bac84da68ac16c1af/users/4f257c8bf6f84581827299ad05b02d4d/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.069684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 943/1880] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:24.069684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bd834-32bd-43b8-a54b-ee640cf3d3e6 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:24.069684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-388bd834-32bd-43b8-a54b-ee640cf3d3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:24.084420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bd834-32bd-43b8-a54b-ee640cf3d3e6 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:24.095800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.098099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.098379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.098652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] PATH_INFO: `/v3/projects/bd6646586b4f4fb1938ac90044d81da0` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.112313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff955c2a-1a20-4eb3-93b2-7291e7ddcbed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df40e616f5f471bac84da68ac16c1af, user_id=4f257c8bf6f84581827299ad05b02d4d, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.115321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff955c2a-1a20-4eb3-93b2-7291e7ddcbed admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:24.115960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd6646586b4f4fb1938ac90044d81da0)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.120273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:24.122599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:24.151817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 938/1881] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:23 2022] PUT /identity/v3/projects/6df40e616f5f471bac84da68ac16c1af/users/4f257c8bf6f84581827299ad05b02d4d/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:24.151817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f91af-1058-4b1b-bf2d-54dc95df90bf None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.151817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f91af-1058-4b1b-bf2d-54dc95df90bf None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.151817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f91af-1058-4b1b-bf2d-54dc95df90bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.191113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-285f91af-1058-4b1b-bf2d-54dc95df90bf None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:24.192442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-285f91af-1058-4b1b-bf2d-54dc95df90bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:24.247760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:24.251605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd6646586b4f4fb1938ac90044d81da0', 'request_id': 'req-1db6d632-e8cd-4801-8713-9c64ddbc6c02'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:24.269350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd6646586b4f4fb1938ac90044d81da0', 'request_id': 'req-1db6d632-e8cd-4801-8713-9c64ddbc6c02'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:24.279962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd6646586b4f4fb1938ac90044d81da0', 'request_id': 'req-1db6d632-e8cd-4801-8713-9c64ddbc6c02'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:24.292878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] The token cache is being invalidate because project bd6646586b4f4fb1938ac90044d81da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:24.293124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:24.293887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1db6d632-e8cd-4801-8713-9c64ddbc6c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:24.299540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 944/1882] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:24 2022] DELETE /identity/v3/projects/bd6646586b4f4fb1938ac90044d81da0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:24.301329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc51746-6931-4e5d-9732-3843235409b1 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.301733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc51746-6931-4e5d-9732-3843235409b1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.302093 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc51746-6931-4e5d-9732-3843235409b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.351013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 939/1883] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:24.352653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febde944-bdd3-42d1-a261-549aa07bca44 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.355752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febde944-bdd3-42d1-a261-549aa07bca44 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.356086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febde944-bdd3-42d1-a261-549aa07bca44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.366601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-8fc51746-6931-4e5d-9732-3843235409b1 None None] MFA Rules not processed for user `4f257c8bf6f84581827299ad05b02d4d`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:24.369069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8fc51746-6931-4e5d-9732-3843235409b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:24.398044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-febde944-bdd3-42d1-a261-549aa07bca44 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:24.399898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-febde944-bdd3-42d1-a261-549aa07bca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:24.459025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 945/1884] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:24.460095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc51746-6931-4e5d-9732-3843235409b1 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:24.514309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 940/1885] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:24.515022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febde944-bdd3-42d1-a261-549aa07bca44 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:24.516329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-febde944-bdd3-42d1-a261-549aa07bca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:24.534954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febde944-bdd3-42d1-a261-549aa07bca44 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:24.541725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d3b58-2793-4cbc-b020-46ef69e77ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.542406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d3b58-2793-4cbc-b020-46ef69e77ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.542689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d3b58-2793-4cbc-b020-46ef69e77ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.542968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d3b58-2793-4cbc-b020-46ef69e77ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.543676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d3b58-2793-4cbc-b020-46ef69e77ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.545782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d3b58-2793-4cbc-b020-46ef69e77ee7 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:24.563653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc51746-6931-4e5d-9732-3843235409b1 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:24.571888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ca624-0b1b-416d-b96e-28e7ecd96682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.573186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca624-0b1b-416d-b96e-28e7ecd96682 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.573467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca624-0b1b-416d-b96e-28e7ecd96682 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.573958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca624-0b1b-416d-b96e-28e7ecd96682 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.574961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ca624-0b1b-416d-b96e-28e7ecd96682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:24.575308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ca624-0b1b-416d-b96e-28e7ecd96682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.577187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ca624-0b1b-416d-b96e-28e7ecd96682 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:24.586452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 941/1886] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:24 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:24.588923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a7fcc-3f61-4c7e-a116-64124e2395de None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.589447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a7fcc-3f61-4c7e-a116-64124e2395de None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.589973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a7fcc-3f61-4c7e-a116-64124e2395de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.598768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 946/1887] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:24.598768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ca624-0b1b-416d-b96e-28e7ecd96682 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:24.608882 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ca624-0b1b-416d-b96e-28e7ecd96682 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:24.619443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-923a7fcc-3f61-4c7e-a116-64124e2395de None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:24.621058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-923a7fcc-3f61-4c7e-a116-64124e2395de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:24.683200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff988f65-23e5-4a52-9c20-2524dab70e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.684369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff988f65-23e5-4a52-9c20-2524dab70e16 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.684589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff988f65-23e5-4a52-9c20-2524dab70e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.684764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff988f65-23e5-4a52-9c20-2524dab70e16 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.685469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff988f65-23e5-4a52-9c20-2524dab70e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.687766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff988f65-23e5-4a52-9c20-2524dab70e16 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:24.696795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:24.699193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 942/1888] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:24.700679 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a7fcc-3f61-4c7e-a116-64124e2395de None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:24.713875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a7fcc-3f61-4c7e-a116-64124e2395de None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:24.722811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ec16d-854f-4079-98f1-09d4d7b1a4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.725930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 947/1889] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:24 2022] POST /identity/v3/users => generated 296 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:24.726472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff988f65-23e5-4a52-9c20-2524dab70e16 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:24.727664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ec16d-854f-4079-98f1-09d4d7b1a4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.727855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ec16d-854f-4079-98f1-09d4d7b1a4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.728319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ec16d-854f-4079-98f1-09d4d7b1a4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.728846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ec16d-854f-4079-98f1-09d4d7b1a4b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.730833 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ec16d-854f-4079-98f1-09d4d7b1a4b4 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:24.741611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff988f65-23e5-4a52-9c20-2524dab70e16 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:24.745915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:24.751499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9c5e5-07a1-454a-b074-4b676eec7091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.753018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9c5e5-07a1-454a-b074-4b676eec7091 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.753409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9c5e5-07a1-454a-b074-4b676eec7091 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.754043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9c5e5-07a1-454a-b074-4b676eec7091 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.754759 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9c5e5-07a1-454a-b074-4b676eec7091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:24.755402 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9c5e5-07a1-454a-b074-4b676eec7091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.757977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9c5e5-07a1-454a-b074-4b676eec7091 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:24.772015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 943/1890] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:24 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:24.773403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ec16d-854f-4079-98f1-09d4d7b1a4b4 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:24.775874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 948/1891] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:24.776767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9c5e5-07a1-454a-b074-4b676eec7091 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:24.788756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9c5e5-07a1-454a-b074-4b676eec7091 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:24.859787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ec16d-854f-4079-98f1-09d4d7b1a4b4 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:24.868589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e5c5-d1d6-4455-8743-eaf1edba98d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.869455 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96bb14-3d1e-4efe-a4dd-7080326f12ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.869816 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e5c5-d1d6-4455-8743-eaf1edba98d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.870272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e5c5-d1d6-4455-8743-eaf1edba98d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.870409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96bb14-3d1e-4efe-a4dd-7080326f12ce admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.870799 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96bb14-3d1e-4efe-a4dd-7080326f12ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.870979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198e5c5-d1d6-4455-8743-eaf1edba98d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.871249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96bb14-3d1e-4efe-a4dd-7080326f12ce admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.872040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96bb14-3d1e-4efe-a4dd-7080326f12ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.872840 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198e5c5-d1d6-4455-8743-eaf1edba98d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.874354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96bb14-3d1e-4efe-a4dd-7080326f12ce admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:24.874981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198e5c5-d1d6-4455-8743-eaf1edba98d4 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:24.887626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 944/1892] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:24.888745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e5c5-d1d6-4455-8743-eaf1edba98d4 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:24.902213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198e5c5-d1d6-4455-8743-eaf1edba98d4 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:24.911517 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 949/1893] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:24 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:24.918038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b4d5b-2a42-4394-834e-5fa56d5d204e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.921364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96bb14-3d1e-4efe-a4dd-7080326f12ce admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:24.922058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b4d5b-2a42-4394-834e-5fa56d5d204e admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.923033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b4d5b-2a42-4394-834e-5fa56d5d204e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.923324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b4d5b-2a42-4394-834e-5fa56d5d204e admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.924057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b4d5b-2a42-4394-834e-5fa56d5d204e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.928537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b4d5b-2a42-4394-834e-5fa56d5d204e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:24.942370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96bb14-3d1e-4efe-a4dd-7080326f12ce admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:24.949224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b0ff4-407c-4fbe-8dd8-545a45d28442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:24.949932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b0ff4-407c-4fbe-8dd8-545a45d28442 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:24.950328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b0ff4-407c-4fbe-8dd8-545a45d28442 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:24.950606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b0ff4-407c-4fbe-8dd8-545a45d28442 admin admin] PATH_INFO: `/v3/projects/63dfad47e8f14958a5724e98cd30b452/users/80c52e789bb54db48de5fb4848f476f3/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:24.984987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b0ff4-407c-4fbe-8dd8-545a45d28442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63dfad47e8f14958a5724e98cd30b452, user_id=80c52e789bb54db48de5fb4848f476f3, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:24.990427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b0ff4-407c-4fbe-8dd8-545a45d28442 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.021928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 945/1894] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:24 2022] POST /identity/v3/projects => generated 393 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:25.022623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b4d5b-2a42-4394-834e-5fa56d5d204e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.027387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 950/1895] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:24 2022] PUT /identity/v3/projects/63dfad47e8f14958a5724e98cd30b452/users/80c52e789bb54db48de5fb4848f476f3/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:25.029465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b0ff4-407c-4fbe-8dd8-545a45d28442 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.117454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b4d5b-2a42-4394-834e-5fa56d5d204e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.124561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaca6f8-c1aa-476a-bf96-32c41ed7636e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.125497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaca6f8-c1aa-476a-bf96-32c41ed7636e admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.126066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaca6f8-c1aa-476a-bf96-32c41ed7636e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.126424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaca6f8-c1aa-476a-bf96-32c41ed7636e admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.127290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaca6f8-c1aa-476a-bf96-32c41ed7636e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.129068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaca6f8-c1aa-476a-bf96-32c41ed7636e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.146153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b0ff4-407c-4fbe-8dd8-545a45d28442 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.155476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da902d1b-6ea5-48a1-b815-2846e2ce8b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.156162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da902d1b-6ea5-48a1-b815-2846e2ce8b6e admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.156493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da902d1b-6ea5-48a1-b815-2846e2ce8b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.156772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da902d1b-6ea5-48a1-b815-2846e2ce8b6e admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.157677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da902d1b-6ea5-48a1-b815-2846e2ce8b6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.159812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da902d1b-6ea5-48a1-b815-2846e2ce8b6e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.180916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 946/1896] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:25.187765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 951/1897] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:25 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:25.188616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da902d1b-6ea5-48a1-b815-2846e2ce8b6e admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.205622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaca6f8-c1aa-476a-bf96-32c41ed7636e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.214288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da902d1b-6ea5-48a1-b815-2846e2ce8b6e admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.220159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749c886-8b5c-4d99-9fe4-9bd97ad6b9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.221206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749c886-8b5c-4d99-9fe4-9bd97ad6b9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.221825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749c886-8b5c-4d99-9fe4-9bd97ad6b9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.222258 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749c886-8b5c-4d99-9fe4-9bd97ad6b9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.225163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749c886-8b5c-4d99-9fe4-9bd97ad6b9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.227336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749c886-8b5c-4d99-9fe4-9bd97ad6b9b4 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.233035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaca6f8-c1aa-476a-bf96-32c41ed7636e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.245124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 952/1898] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:25.247077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749c886-8b5c-4d99-9fe4-9bd97ad6b9b4 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.270195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4f43-ee1c-4fde-b669-3fba45587d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.271251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4f43-ee1c-4fde-b669-3fba45587d96 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.271608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4f43-ee1c-4fde-b669-3fba45587d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.272021 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4f43-ee1c-4fde-b669-3fba45587d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.273162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c4f43-ee1c-4fde-b669-3fba45587d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.273650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749c886-8b5c-4d99-9fe4-9bd97ad6b9b4 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.276332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c4f43-ee1c-4fde-b669-3fba45587d96 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.299969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b27ec-ef59-41eb-8c91-30212277883f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.301621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b27ec-ef59-41eb-8c91-30212277883f admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.301621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b27ec-ef59-41eb-8c91-30212277883f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.301621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b27ec-ef59-41eb-8c91-30212277883f admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.301990 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b27ec-ef59-41eb-8c91-30212277883f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.304667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b27ec-ef59-41eb-8c91-30212277883f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.317861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 947/1899] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:25.318826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4f43-ee1c-4fde-b669-3fba45587d96 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.349231 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4f43-ee1c-4fde-b669-3fba45587d96 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.350884 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 953/1900] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:25.352146 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b27ec-ef59-41eb-8c91-30212277883f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.373337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6facf78-6515-4a6a-ad40-41927cee34d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.375357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6facf78-6515-4a6a-ad40-41927cee34d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.375357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6facf78-6515-4a6a-ad40-41927cee34d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.375549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6facf78-6515-4a6a-ad40-41927cee34d4 admin admin] PATH_INFO: `/v3/projects/63dfad47e8f14958a5724e98cd30b452/users/80c52e789bb54db48de5fb4848f476f3/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.382295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b27ec-ef59-41eb-8c91-30212277883f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.384767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6facf78-6515-4a6a-ad40-41927cee34d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63dfad47e8f14958a5724e98cd30b452, user_id=80c52e789bb54db48de5fb4848f476f3, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.387654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbbebf-511c-4e1b-a9d3-1e44ad4c1900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.389534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbbebf-511c-4e1b-a9d3-1e44ad4c1900 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.390004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbbebf-511c-4e1b-a9d3-1e44ad4c1900 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.390311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbbebf-511c-4e1b-a9d3-1e44ad4c1900 admin admin] PATH_INFO: `/v3/projects/e736630e54504108b00dac22688c2055/users/8a88bef44e2d4988a6bfa88bd82d8c10/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.392307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6facf78-6515-4a6a-ad40-41927cee34d4 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.395304 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:INSERT=10,keystone:SELECT=90 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:25.427279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 948/1901] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:25 2022] PUT /identity/v3/projects/63dfad47e8f14958a5724e98cd30b452/users/80c52e789bb54db48de5fb4848f476f3/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:25.429662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6facf78-6515-4a6a-ad40-41927cee34d4 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.433490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbbebf-511c-4e1b-a9d3-1e44ad4c1900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e736630e54504108b00dac22688c2055, user_id=8a88bef44e2d4988a6bfa88bd82d8c10, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.438278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbbebf-511c-4e1b-a9d3-1e44ad4c1900 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.471673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 954/1902] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:25 2022] PUT /identity/v3/projects/e736630e54504108b00dac22688c2055/users/8a88bef44e2d4988a6bfa88bd82d8c10/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:25.475703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb693a9-b6ad-4a32-a4a8-dfe31d922d65 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.475920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb693a9-b6ad-4a32-a4a8-dfe31d922d65 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.476207 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb693a9-b6ad-4a32-a4a8-dfe31d922d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.520717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-bbb693a9-b6ad-4a32-a4a8-dfe31d922d65 None None] MFA Rules not processed for user `80c52e789bb54db48de5fb4848f476f3`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:25.524151 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-bbb693a9-b6ad-4a32-a4a8-dfe31d922d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:25.559636 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:25.606247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6facf78-6515-4a6a-ad40-41927cee34d4 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.651059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 955/1903] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:25.653042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c515c-74f8-4364-ad16-ba518c57c1fc None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.653549 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c515c-74f8-4364-ad16-ba518c57c1fc None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.654234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c515c-74f8-4364-ad16-ba518c57c1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.705320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-2f3c515c-74f8-4364-ad16-ba518c57c1fc None None] MFA Rules not processed for user `8a88bef44e2d4988a6bfa88bd82d8c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:25.707686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2f3c515c-74f8-4364-ad16-ba518c57c1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:25.730622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eb44b-41d8-4a20-8f66-c42f151b5f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.731405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb44b-41d8-4a20-8f66-c42f151b5f11 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.731699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb44b-41d8-4a20-8f66-c42f151b5f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.731980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eb44b-41d8-4a20-8f66-c42f151b5f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.732724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eb44b-41d8-4a20-8f66-c42f151b5f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.734369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eb44b-41d8-4a20-8f66-c42f151b5f11 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.752443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 949/1904] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:25.753334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eb44b-41d8-4a20-8f66-c42f151b5f11 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.755119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-260eb44b-41d8-4a20-8f66-c42f151b5f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:25.831811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 956/1905] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:25.832659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c515c-74f8-4364-ad16-ba518c57c1fc None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.845274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c515c-74f8-4364-ad16-ba518c57c1fc None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.849747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eb44b-41d8-4a20-8f66-c42f151b5f11 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.850721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cbaeb-148c-4303-8ab7-6c65fbeb6501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.851914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbaeb-148c-4303-8ab7-6c65fbeb6501 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.852241 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbaeb-148c-4303-8ab7-6c65fbeb6501 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.852575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbaeb-148c-4303-8ab7-6c65fbeb6501 admin admin] PATH_INFO: `/v3/projects/0c017416f9d24837be210629a4bd94cd/users/30f1601a2cb4455393fe4f981858857d/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.853933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3962d8-fbaf-48a5-afd3-70d3f9735dce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.854656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3962d8-fbaf-48a5-afd3-70d3f9735dce service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.855034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3962d8-fbaf-48a5-afd3-70d3f9735dce service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.855420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3962d8-fbaf-48a5-afd3-70d3f9735dce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.866864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3962d8-fbaf-48a5-afd3-70d3f9735dce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.868425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3962d8-fbaf-48a5-afd3-70d3f9735dce service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.871485 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cbaeb-148c-4303-8ab7-6c65fbeb6501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c017416f9d24837be210629a4bd94cd, user_id=30f1601a2cb4455393fe4f981858857d, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.875797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cbaeb-148c-4303-8ab7-6c65fbeb6501 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:25.879726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 950/1906] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:25.891503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3962d8-fbaf-48a5-afd3-70d3f9735dce service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:25.901639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 957/1907] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:25 2022] PUT /identity/v3/projects/0c017416f9d24837be210629a4bd94cd/users/30f1601a2cb4455393fe4f981858857d/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:25.910356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d1de0-f357-4072-ab5f-404b973e5a88 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.911203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d1de0-f357-4072-ab5f-404b973e5a88 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.911743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d1de0-f357-4072-ab5f-404b973e5a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.917788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3962d8-fbaf-48a5-afd3-70d3f9735dce service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:25.983840 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33660990-5127-4e10-a2e1-79e7865c4b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:25.984430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33660990-5127-4e10-a2e1-79e7865c4b0d admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:25.984712 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33660990-5127-4e10-a2e1-79e7865c4b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:25.985392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33660990-5127-4e10-a2e1-79e7865c4b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:25.985711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33660990-5127-4e10-a2e1-79e7865c4b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:25.994756 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33660990-5127-4e10-a2e1-79e7865c4b0d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:26.036352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ec9d1de0-f357-4072-ab5f-404b973e5a88 None None] MFA Rules not processed for user `30f1601a2cb4455393fe4f981858857d`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:26.045952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ec9d1de0-f357-4072-ab5f-404b973e5a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:26.089385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 951/1908] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:25 2022] POST /identity/v3/projects => generated 389 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:26.091451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b06c12-7636-4f3e-911e-8d0f60a27fab None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:26.091734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b06c12-7636-4f3e-911e-8d0f60a27fab None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:26.092015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b06c12-7636-4f3e-911e-8d0f60a27fab None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:26.093669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 952/1909] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:53:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:53:26.094382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b06c12-7636-4f3e-911e-8d0f60a27fab None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:26.107110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:26.134661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:26.225977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 958/1910] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:53:26.228107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d1de0-f357-4072-ab5f-404b973e5a88 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:26.263671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d1de0-f357-4072-ab5f-404b973e5a88 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:26.332880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b06c12-7636-4f3e-911e-8d0f60a27fab None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:26.348051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ee72f-dc14-4482-b846-d550d8230bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:26.351953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ee72f-dc14-4482-b846-d550d8230bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:26.351953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ee72f-dc14-4482-b846-d550d8230bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:26.351953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ee72f-dc14-4482-b846-d550d8230bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:26.351953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ee72f-dc14-4482-b846-d550d8230bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:26.356294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ee72f-dc14-4482-b846-d550d8230bd8 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:26.411406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 953/1911] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:26 2022] POST /identity/v3/users => generated 301 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:26.414017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967b421-a313-4890-bce0-13a87ea3cd08 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:26.414460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967b421-a313-4890-bce0-13a87ea3cd08 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:26.414705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967b421-a313-4890-bce0-13a87ea3cd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:26.433877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e9629-7c17-45a0-bd69-0e7704e58aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:26.436509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9629-7c17-45a0-bd69-0e7704e58aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:26.436986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9629-7c17-45a0-bd69-0e7704e58aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:26.438469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9629-7c17-45a0-bd69-0e7704e58aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:26.442376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e9629-7c17-45a0-bd69-0e7704e58aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:26.447907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e9629-7c17-45a0-bd69-0e7704e58aa3 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:26.469760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-b967b421-a313-4890-bce0-13a87ea3cd08 None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:26.471095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b967b421-a313-4890-bce0-13a87ea3cd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:26.500540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 959/1912] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:26 2022] POST /identity/v3/projects => generated 391 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:26.500540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e9629-7c17-45a0-bd69-0e7704e58aa3 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:26.604862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e9629-7c17-45a0-bd69-0e7704e58aa3 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:26.619818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd97d7-6deb-4dc3-b4f8-9b3ac1fb8879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:26.621296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd97d7-6deb-4dc3-b4f8-9b3ac1fb8879 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:26.622295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd97d7-6deb-4dc3-b4f8-9b3ac1fb8879 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:26.622794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd97d7-6deb-4dc3-b4f8-9b3ac1fb8879 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:26.625347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd97d7-6deb-4dc3-b4f8-9b3ac1fb8879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:26.628644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd97d7-6deb-4dc3-b4f8-9b3ac1fb8879 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:26.644628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 960/1913] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:26.646242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd97d7-6deb-4dc3-b4f8-9b3ac1fb8879 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:26.650444 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 954/1914] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:53:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:26.651245 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967b421-a313-4890-bce0-13a87ea3cd08 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:26.663039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd97d7-6deb-4dc3-b4f8-9b3ac1fb8879 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:26.663930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967b421-a313-4890-bce0-13a87ea3cd08 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:26.669632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122094b-11d7-4f20-80c1-7032b533f38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:26.670895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122094b-11d7-4f20-80c1-7032b533f38a admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:26.671403 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122094b-11d7-4f20-80c1-7032b533f38a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:26.671897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122094b-11d7-4f20-80c1-7032b533f38a admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:26.672914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122094b-11d7-4f20-80c1-7032b533f38a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:26.673043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3c35d-f91f-4d63-891d-19d323da1b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:26.674229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3c35d-f91f-4d63-891d-19d323da1b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:26.674515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3c35d-f91f-4d63-891d-19d323da1b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:26.674834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3c35d-f91f-4d63-891d-19d323da1b87 admin admin] PATH_INFO: `/v3/projects/c37ae2b07c9b4bb2a06641ba79ae478b/users/72a2b97fbe2e44beb66fc35d8e4abe1b/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:26.677134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122094b-11d7-4f20-80c1-7032b533f38a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:26.705968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3c35d-f91f-4d63-891d-19d323da1b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37ae2b07c9b4bb2a06641ba79ae478b, user_id=72a2b97fbe2e44beb66fc35d8e4abe1b, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:26.710214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3c35d-f91f-4d63-891d-19d323da1b87 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:26.712730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 961/1915] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:26 2022] POST /identity/v3/users => generated 302 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:26.714001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122094b-11d7-4f20-80c1-7032b533f38a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:26.755903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 955/1916] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:26 2022] PUT /identity/v3/projects/c37ae2b07c9b4bb2a06641ba79ae478b/users/72a2b97fbe2e44beb66fc35d8e4abe1b/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:26.756769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3c35d-f91f-4d63-891d-19d323da1b87 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:26.775374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f3c3c35d-f91f-4d63-891d-19d323da1b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:26.825471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:26.873490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122094b-11d7-4f20-80c1-7032b533f38a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:26.945575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6022480-a07b-4304-8e88-a9a505a358a1 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80c52e789bb54db48de5fb4848f476f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dfad47e8f14958a5724e98cd30b452', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:26.948488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6022480-a07b-4304-8e88-a9a505a358a1 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:26.948488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6022480-a07b-4304-8e88-a9a505a358a1 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:26.948488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6022480-a07b-4304-8e88-a9a505a358a1 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:26.949108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6022480-a07b-4304-8e88-a9a505a358a1 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:26.949474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6022480-a07b-4304-8e88-a9a505a358a1 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:26.951271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6022480-a07b-4304-8e88-a9a505a358a1 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:26.952278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3c35d-f91f-4d63-891d-19d323da1b87 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:26.975207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a0773-03e9-4c14-9c55-46da3a0ee5cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:26.975207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a0773-03e9-4c14-9c55-46da3a0ee5cd service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:26.975207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a0773-03e9-4c14-9c55-46da3a0ee5cd service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:26.975207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a0773-03e9-4c14-9c55-46da3a0ee5cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:26.985119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:26.985762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 962/1917] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:26.987607 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6022480-a07b-4304-8e88-a9a505a358a1 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:26.989773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a0773-03e9-4c14-9c55-46da3a0ee5cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:26.991499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a0773-03e9-4c14-9c55-46da3a0ee5cd service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.031315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 956/1918] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:27.032630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a0773-03e9-4c14-9c55-46da3a0ee5cd service nova] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:27.164413 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a0773-03e9-4c14-9c55-46da3a0ee5cd service nova] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:27.178051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0735d62-ec1a-4327-a4d1-374f0c47b4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:27.178738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0735d62-ec1a-4327-a4d1-374f0c47b4ac admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.179611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0735d62-ec1a-4327-a4d1-374f0c47b4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.180118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0735d62-ec1a-4327-a4d1-374f0c47b4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.181534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0735d62-ec1a-4327-a4d1-374f0c47b4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:27.182671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6022480-a07b-4304-8e88-a9a505a358a1 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:27.184936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0735d62-ec1a-4327-a4d1-374f0c47b4ac admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.204846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 957/1919] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:27.205058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4270a-cb38-46c7-a9f3-0a7279130e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:27.205772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4270a-cb38-46c7-a9f3-0a7279130e4a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.206057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4270a-cb38-46c7-a9f3-0a7279130e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.206333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4270a-cb38-46c7-a9f3-0a7279130e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.206442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0735d62-ec1a-4327-a4d1-374f0c47b4ac admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:27.207076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4270a-cb38-46c7-a9f3-0a7279130e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:27.208835 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4270a-cb38-46c7-a9f3-0a7279130e4a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.222891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0735d62-ec1a-4327-a4d1-374f0c47b4ac admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:27.238824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f998ae-3386-49a1-9aa8-6b172d0496b4 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80c52e789bb54db48de5fb4848f476f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dfad47e8f14958a5724e98cd30b452', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:27.240257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f998ae-3386-49a1-9aa8-6b172d0496b4 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.240675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f998ae-3386-49a1-9aa8-6b172d0496b4 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.240967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f998ae-3386-49a1-9aa8-6b172d0496b4 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.242592 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f998ae-3386-49a1-9aa8-6b172d0496b4 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:27.245637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 963/1920] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:27.246124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f998ae-3386-49a1-9aa8-6b172d0496b4 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.246597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4270a-cb38-46c7-a9f3-0a7279130e4a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:27.262288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4270a-cb38-46c7-a9f3-0a7279130e4a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:27.268210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcb2d0-c095-4490-9339-6c42f9d20483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:27.269225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcb2d0-c095-4490-9339-6c42f9d20483 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.270499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcb2d0-c095-4490-9339-6c42f9d20483 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.270499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcb2d0-c095-4490-9339-6c42f9d20483 admin admin] PATH_INFO: `/v3/projects/c37ae2b07c9b4bb2a06641ba79ae478b/users/72a2b97fbe2e44beb66fc35d8e4abe1b/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.272695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcb2d0-c095-4490-9339-6c42f9d20483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37ae2b07c9b4bb2a06641ba79ae478b, user_id=72a2b97fbe2e44beb66fc35d8e4abe1b, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:27.276596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcb2d0-c095-4490-9339-6c42f9d20483 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.293166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 964/1921] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:27 2022] PUT /identity/v3/projects/c37ae2b07c9b4bb2a06641ba79ae478b/users/72a2b97fbe2e44beb66fc35d8e4abe1b/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:27.294120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcb2d0-c095-4490-9339-6c42f9d20483 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:27.304587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 958/1922] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:27 2022] POST /identity/v3/projects => generated 354 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:27.307101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d06bf7-9ad4-4abb-9114-cbd0bcd41d0d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.307579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d06bf7-9ad4-4abb-9114-cbd0bcd41d0d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.308106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d06bf7-9ad4-4abb-9114-cbd0bcd41d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.354919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-48d06bf7-9ad4-4abb-9114-cbd0bcd41d0d None None] MFA Rules not processed for user `72a2b97fbe2e44beb66fc35d8e4abe1b`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:27.356314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-48d06bf7-9ad4-4abb-9114-cbd0bcd41d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:27.388444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:27.402066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcb2d0-c095-4490-9339-6c42f9d20483 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:27.445186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 959/1923] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:27.446885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d06bf7-9ad4-4abb-9114-cbd0bcd41d0d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:27.472505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb43b74-6fd4-4903-8643-e239b80950d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:27.473670 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb43b74-6fd4-4903-8643-e239b80950d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.474233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb43b74-6fd4-4903-8643-e239b80950d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.474346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb43b74-6fd4-4903-8643-e239b80950d8 admin admin] PATH_INFO: `/v3/projects/3d6e456327d7419583b4d47a2cc94e6f/users/08f1991e453b4945b72f66643a4c3b27/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.491124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb43b74-6fd4-4903-8643-e239b80950d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6e456327d7419583b4d47a2cc94e6f, user_id=08f1991e453b4945b72f66643a4c3b27, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:27.494582 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb43b74-6fd4-4903-8643-e239b80950d8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.516800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:27.519246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 965/1924] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:27 2022] PUT /identity/v3/projects/3d6e456327d7419583b4d47a2cc94e6f/users/08f1991e453b4945b72f66643a4c3b27/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:27.520241 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb43b74-6fd4-4903-8643-e239b80950d8 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:27.523690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-efb43b74-6fd4-4903-8643-e239b80950d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:27.536091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d06bf7-9ad4-4abb-9114-cbd0bcd41d0d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:27.608536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb43b74-6fd4-4903-8643-e239b80950d8 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:27.613053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4aa41-cfc7-4fab-a4e9-7964c65df3da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:27.613748 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4aa41-cfc7-4fab-a4e9-7964c65df3da service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.614170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4aa41-cfc7-4fab-a4e9-7964c65df3da service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.614719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4aa41-cfc7-4fab-a4e9-7964c65df3da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.633468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5c890-6ef9-438b-b272-37df121aca53 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80c52e789bb54db48de5fb4848f476f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dfad47e8f14958a5724e98cd30b452', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:27.635200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5c890-6ef9-438b-b272-37df121aca53 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.635889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5c890-6ef9-438b-b272-37df121aca53 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.636352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5c890-6ef9-438b-b272-37df121aca53 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.637929 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5c890-6ef9-438b-b272-37df121aca53 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:27.638442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5c890-6ef9-438b-b272-37df121aca53 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:27.640470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5c890-6ef9-438b-b272-37df121aca53 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.660047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 960/1925] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:27.661487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5c890-6ef9-438b-b272-37df121aca53 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:27.697828 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4aa41-cfc7-4fab-a4e9-7964c65df3da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:27.699599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4aa41-cfc7-4fab-a4e9-7964c65df3da service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.731322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 966/1926] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:27.732746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4aa41-cfc7-4fab-a4e9-7964c65df3da service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:27.742732 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5c890-6ef9-438b-b272-37df121aca53 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:27.746387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4aa41-cfc7-4fab-a4e9-7964c65df3da service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:27.747836 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5937f6a-8305-45ba-93fe-0b11fe77da90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:27.748508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5937f6a-8305-45ba-93fe-0b11fe77da90 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.748785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5937f6a-8305-45ba-93fe-0b11fe77da90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.749052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5937f6a-8305-45ba-93fe-0b11fe77da90 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.749883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5937f6a-8305-45ba-93fe-0b11fe77da90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:27.758279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5937f6a-8305-45ba-93fe-0b11fe77da90 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.771975 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80c52e789bb54db48de5fb4848f476f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '63dfad47e8f14958a5724e98cd30b452', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:27.775723 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 961/1927] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:27.776515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.776645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5937f6a-8305-45ba-93fe-0b11fe77da90 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:27.778243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.779874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] PATH_INFO: `/v3/projects/22dc7d81bc614706981be2b979cb034d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.783616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: Authorizing `identity:delete_project(project_id=22dc7d81bc614706981be2b979cb034d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:27.790927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.792105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5937f6a-8305-45ba-93fe-0b11fe77da90 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:27.797288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c28e28-e050-4e4a-b005-b556ecbc0253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:27.798260 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c28e28-e050-4e4a-b005-b556ecbc0253 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.798676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c28e28-e050-4e4a-b005-b556ecbc0253 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.798955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c28e28-e050-4e4a-b005-b556ecbc0253 admin admin] PATH_INFO: `/v3/projects/3d6e456327d7419583b4d47a2cc94e6f/users/08f1991e453b4945b72f66643a4c3b27/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.801842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c28e28-e050-4e4a-b005-b556ecbc0253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6e456327d7419583b4d47a2cc94e6f, user_id=08f1991e453b4945b72f66643a4c3b27, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:27.804896 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c28e28-e050-4e4a-b005-b556ecbc0253 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:27.825097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 962/1928] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:27 2022] PUT /identity/v3/projects/3d6e456327d7419583b4d47a2cc94e6f/users/08f1991e453b4945b72f66643a4c3b27/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:27.839012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a1468-b86e-47a8-a41e-ff7ad07090ae None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:27.840497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a1468-b86e-47a8-a41e-ff7ad07090ae None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:27.840976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a1468-b86e-47a8-a41e-ff7ad07090ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:27.868739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:27.898613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-fe2a1468-b86e-47a8-a41e-ff7ad07090ae None None] MFA Rules not processed for user `08f1991e453b4945b72f66643a4c3b27`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:27.900712 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fe2a1468-b86e-47a8-a41e-ff7ad07090ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:27.914911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22dc7d81bc614706981be2b979cb034d', 'request_id': 'req-c9bb3218-a141-4534-bb25-22b0c53a3b74'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:27.926442 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22dc7d81bc614706981be2b979cb034d', 'request_id': 'req-c9bb3218-a141-4534-bb25-22b0c53a3b74'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:27.935803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22dc7d81bc614706981be2b979cb034d', 'request_id': 'req-c9bb3218-a141-4534-bb25-22b0c53a3b74'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:27.958338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] The token cache is being invalidate because project 22dc7d81bc614706981be2b979cb034d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:27.958741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:27.960300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:27.969126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 967/1929] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:27 2022] DELETE /identity/v3/projects/22dc7d81bc614706981be2b979cb034d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:27.970716 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:27.973134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.020628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 963/1930] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:28.034058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a1468-b86e-47a8-a41e-ff7ad07090ae None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:28.038311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fe2a1468-b86e-47a8-a41e-ff7ad07090ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.047764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:28.114392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb3218-a141-4534-bb25-22b0c53a3b74 tempest-VolumeQuotaClassesTest-720262043 tempest-VolumeQuotaClassesTest-720262043-project-admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:28.122657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:28.124068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.124309 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.124789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] PATH_INFO: `/v3/users/8a88bef44e2d4988a6bfa88bd82d8c10` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.126315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a88bef44e2d4988a6bfa88bd82d8c10)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:28.130139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:28.147272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a1468-b86e-47a8-a41e-ff7ad07090ae None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:28.158969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec8f58-7511-4b1a-9f7f-a5d4a58e6e1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:28.159627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec8f58-7511-4b1a-9f7f-a5d4a58e6e1a service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.159914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec8f58-7511-4b1a-9f7f-a5d4a58e6e1a service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.160189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec8f58-7511-4b1a-9f7f-a5d4a58e6e1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.185033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec8f58-7511-4b1a-9f7f-a5d4a58e6e1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:28.186926 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec8f58-7511-4b1a-9f7f-a5d4a58e6e1a service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:28.191422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a88bef44e2d4988a6bfa88bd82d8c10'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.198586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 964/1931] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:53:28.202346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a88bef44e2d4988a6bfa88bd82d8c10'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.204134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a88bef44e2d4988a6bfa88bd82d8c10'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.219561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a88bef44e2d4988a6bfa88bd82d8c10'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.220769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a88bef44e2d4988a6bfa88bd82d8c10'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.249908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 968/1932] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:27 2022] DELETE /identity/v3/users/8a88bef44e2d4988a6bfa88bd82d8c10 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:28.252241 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d30be9-82a1-4d27-89dd-26a9772e5d07 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.252608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d30be9-82a1-4d27-89dd-26a9772e5d07 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.252968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d30be9-82a1-4d27-89dd-26a9772e5d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.267079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:28.270949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.288219 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-03d30be9-82a1-4d27-89dd-26a9772e5d07 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:28.290665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-03d30be9-82a1-4d27-89dd-26a9772e5d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.328769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:28.364290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f007e-1334-4166-9200-ab8c894fd926 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:28.369281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:28.370619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.370739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.371055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] PATH_INFO: `/v3/users/4f257c8bf6f84581827299ad05b02d4d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.373099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f257c8bf6f84581827299ad05b02d4d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:28.375170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:28.379529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 965/1933] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:28.380294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d30be9-82a1-4d27-89dd-26a9772e5d07 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:28.381670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-03d30be9-82a1-4d27-89dd-26a9772e5d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.448603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f257c8bf6f84581827299ad05b02d4d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.456558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f257c8bf6f84581827299ad05b02d4d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.457210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f257c8bf6f84581827299ad05b02d4d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.458656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d30be9-82a1-4d27-89dd-26a9772e5d07 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:28.463329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d64091-82a3-4152-980c-253217c5e799 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:28.464006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d64091-82a3-4152-980c-253217c5e799 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.464291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d64091-82a3-4152-980c-253217c5e799 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.464569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d64091-82a3-4152-980c-253217c5e799 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.466435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-29d64091-82a3-4152-980c-253217c5e799 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.470319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f257c8bf6f84581827299ad05b02d4d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.470978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f257c8bf6f84581827299ad05b02d4d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.483857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 969/1934] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:28 2022] DELETE /identity/v3/users/4f257c8bf6f84581827299ad05b02d4d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:28.484669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:28.486008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.530684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d64091-82a3-4152-980c-253217c5e799 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:28.532042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d64091-82a3-4152-980c-253217c5e799 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:28.536590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:28.543323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 966/1935] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:28.544347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d64091-82a3-4152-980c-253217c5e799 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:28.545700 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-29d64091-82a3-4152-980c-253217c5e799 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.553443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841cde3-c99b-4525-952d-1b3b8be277de admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:28.560526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:28.561408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.561728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.561979 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] PATH_INFO: `/v3/users/30ecd48609d74d7a823222f06c4ad7d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.563192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ecd48609d74d7a823222f06c4ad7d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:28.565115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:28.614711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ecd48609d74d7a823222f06c4ad7d5'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.618555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d64091-82a3-4152-980c-253217c5e799 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:28.625042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ecd48609d74d7a823222f06c4ad7d5'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.626281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ecd48609d74d7a823222f06c4ad7d5'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.628891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:28.644068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ecd48609d74d7a823222f06c4ad7d5'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.645233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7fa9bdcd-ffc4-4208-af40-29f8e07632d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ecd48609d74d7a823222f06c4ad7d5'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:28.661227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 970/1936] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:28 2022] DELETE /identity/v3/users/30ecd48609d74d7a823222f06c4ad7d5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:28.663526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b732232-a16f-4a16-a6c6-2647a0e55046 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.663908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b732232-a16f-4a16-a6c6-2647a0e55046 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.664278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b732232-a16f-4a16-a6c6-2647a0e55046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.687135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-8b732232-a16f-4a16-a6c6-2647a0e55046 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:28.688510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8b732232-a16f-4a16-a6c6-2647a0e55046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.697659 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452058d7-6952-46f0-b5d1-8681211a7758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:28.698521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452058d7-6952-46f0-b5d1-8681211a7758 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.698831 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452058d7-6952-46f0-b5d1-8681211a7758 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.699984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452058d7-6952-46f0-b5d1-8681211a7758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.702525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-452058d7-6952-46f0-b5d1-8681211a7758 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.719841 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452058d7-6952-46f0-b5d1-8681211a7758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:28.722244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452058d7-6952-46f0-b5d1-8681211a7758 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:28.733487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 967/1937] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:28.735090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf87fa-eafa-4480-9861-f90220007a5a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.735438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf87fa-eafa-4480-9861-f90220007a5a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.735930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf87fa-eafa-4480-9861-f90220007a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.757085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-64bf87fa-eafa-4480-9861-f90220007a5a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:28.758647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-64bf87fa-eafa-4480-9861-f90220007a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.770553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 971/1938] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:28.784314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b732232-a16f-4a16-a6c6-2647a0e55046 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:28.786640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8b732232-a16f-4a16-a6c6-2647a0e55046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.846591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 968/1939] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:28.862287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf87fa-eafa-4480-9861-f90220007a5a None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:28.863917 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-64bf87fa-eafa-4480-9861-f90220007a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:28.910800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b732232-a16f-4a16-a6c6-2647a0e55046 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:28.917518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320840eb-6fe2-4995-a4ce-30238d4baf61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:28.918326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320840eb-6fe2-4995-a4ce-30238d4baf61 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.918669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320840eb-6fe2-4995-a4ce-30238d4baf61 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.918987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320840eb-6fe2-4995-a4ce-30238d4baf61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.944825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320840eb-6fe2-4995-a4ce-30238d4baf61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:28.947152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320840eb-6fe2-4995-a4ce-30238d4baf61 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:28.949737 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf87fa-eafa-4480-9861-f90220007a5a None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:28.954075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964c0e8-57b4-4506-8567-c35d575b8c1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:28.954976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964c0e8-57b4-4506-8567-c35d575b8c1f service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:28.955522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964c0e8-57b4-4506-8567-c35d575b8c1f service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:28.955949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964c0e8-57b4-4506-8567-c35d575b8c1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:28.975108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 972/1940] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:28.978741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964c0e8-57b4-4506-8567-c35d575b8c1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:28.980184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964c0e8-57b4-4506-8567-c35d575b8c1f service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:28.990715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 969/1941] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:29.422499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320840eb-6fe2-4995-a4ce-30238d4baf61 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:29.424329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-320840eb-6fe2-4995-a4ce-30238d4baf61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:29.437764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320840eb-6fe2-4995-a4ce-30238d4baf61 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:29.442404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:29.443269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:29.443549 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:29.443817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] PATH_INFO: `/v3/projects/e736630e54504108b00dac22688c2055` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:29.445232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e736630e54504108b00dac22688c2055)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:29.447390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:29.512499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e736630e54504108b00dac22688c2055', 'request_id': 'req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.545445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e736630e54504108b00dac22688c2055', 'request_id': 'req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.559491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e736630e54504108b00dac22688c2055', 'request_id': 'req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.579989 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:29.590352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] The token cache is being invalidate because project e736630e54504108b00dac22688c2055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:29.598052 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.598822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.600666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 973/1942] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:29 2022] DELETE /identity/v3/projects/e736630e54504108b00dac22688c2055 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:29.604507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964c0e8-57b4-4506-8567-c35d575b8c1f service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:29.610586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:29.620388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964c0e8-57b4-4506-8567-c35d575b8c1f service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:29.716428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:29.717627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:29.718159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:29.718615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] PATH_INFO: `/v3/users/72a2b97fbe2e44beb66fc35d8e4abe1b` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:29.720273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a2b97fbe2e44beb66fc35d8e4abe1b)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:29.724983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:29.814702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a2b97fbe2e44beb66fc35d8e4abe1b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.822072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:29.823686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:29.842741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a2b97fbe2e44beb66fc35d8e4abe1b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.847110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2b067-8a88-49c8-97d3-484bf5a6eb6c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:29.852873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:29.854712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:29.855193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:29.855461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] PATH_INFO: `/v3/projects/6df40e616f5f471bac84da68ac16c1af` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:29.857717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df40e616f5f471bac84da68ac16c1af)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:29.858682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a2b97fbe2e44beb66fc35d8e4abe1b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.859873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:29.877976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a2b97fbe2e44beb66fc35d8e4abe1b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.879101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a2b97fbe2e44beb66fc35d8e4abe1b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.890035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 970/1943] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:29 2022] DELETE /identity/v3/users/72a2b97fbe2e44beb66fc35d8e4abe1b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:29.890752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:29.893264 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:29.921366 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df40e616f5f471bac84da68ac16c1af', 'request_id': 'req-b692f4d7-df47-44ff-8985-e8ce49521b23'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.935689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df40e616f5f471bac84da68ac16c1af', 'request_id': 'req-b692f4d7-df47-44ff-8985-e8ce49521b23'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.952420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df40e616f5f471bac84da68ac16c1af', 'request_id': 'req-b692f4d7-df47-44ff-8985-e8ce49521b23'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:29.968669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc36fd-3771-43a0-b24e-e3f50f9780b9 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:30.004470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:30.045043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:30.046098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:30.046548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:30.046866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] PATH_INFO: `/v3/projects/a13abdc4d0524656984430c3aea7e95a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:30.048204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a13abdc4d0524656984430c3aea7e95a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:30.051240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:30.053609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] The token cache is being invalidate because project 6df40e616f5f471bac84da68ac16c1af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:30.053913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.065392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.074574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 974/1944] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:29 2022] DELETE /identity/v3/projects/6df40e616f5f471bac84da68ac16c1af => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:30.075534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:30.098944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:30.123962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692f4d7-df47-44ff-8985-e8ce49521b23 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:30.130701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:30.132026 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:30.132460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:30.133808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] PATH_INFO: `/v3/users/80c52e789bb54db48de5fb4848f476f3` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:30.135597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c52e789bb54db48de5fb4848f476f3)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:30.138121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:30.183910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a13abdc4d0524656984430c3aea7e95a', 'request_id': 'req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.201362 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a13abdc4d0524656984430c3aea7e95a', 'request_id': 'req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.217070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a13abdc4d0524656984430c3aea7e95a', 'request_id': 'req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.229502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c52e789bb54db48de5fb4848f476f3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.247391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c52e789bb54db48de5fb4848f476f3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.248013 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c52e789bb54db48de5fb4848f476f3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.253438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] The token cache is being invalidate because project a13abdc4d0524656984430c3aea7e95a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:30.253765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.255221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.257375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 971/1945] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:29 2022] DELETE /identity/v3/projects/a13abdc4d0524656984430c3aea7e95a => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:30.262723 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:30.264363 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:30.286868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c52e789bb54db48de5fb4848f476f3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.289350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c52e789bb54db48de5fb4848f476f3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.330078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 975/1946] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:30 2022] DELETE /identity/v3/users/80c52e789bb54db48de5fb4848f476f3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:30.376722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c78b8e-6c77-4fc9-8a95-6d28a2ec5c27 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:30.515187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:30.516206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:30.516478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:30.516760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] PATH_INFO: `/v3/users/7f67e5d9abad4fc28bc4ac35681604be` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:30.519973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f67e5d9abad4fc28bc4ac35681604be)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:30.522486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:30.538471 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:30.601494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:30.603439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:30.627404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8781d-8f65-4482-9319-4d8e7ed073f4 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:30.641098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:30.642025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:30.642311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:30.642590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] PATH_INFO: `/v3/projects/c37ae2b07c9b4bb2a06641ba79ae478b` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:30.644542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37ae2b07c9b4bb2a06641ba79ae478b)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:30.647208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f67e5d9abad4fc28bc4ac35681604be'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.656667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:30.672946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f67e5d9abad4fc28bc4ac35681604be'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.700854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f67e5d9abad4fc28bc4ac35681604be'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.714502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f67e5d9abad4fc28bc4ac35681604be'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.715379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c27ef7e9-188e-4f53-9f26-72d554de4f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f67e5d9abad4fc28bc4ac35681604be'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.724730 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 972/1947] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:30 2022] DELETE /identity/v3/users/7f67e5d9abad4fc28bc4ac35681604be => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:30.744731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37ae2b07c9b4bb2a06641ba79ae478b', 'request_id': 'req-002a0513-ac9e-4db5-95ab-2d25261aaa08'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.746868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=77,keystone:DELETE=21,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:30.763274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37ae2b07c9b4bb2a06641ba79ae478b', 'request_id': 'req-002a0513-ac9e-4db5-95ab-2d25261aaa08'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.772775 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37ae2b07c9b4bb2a06641ba79ae478b', 'request_id': 'req-002a0513-ac9e-4db5-95ab-2d25261aaa08'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.812004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] The token cache is being invalidate because project c37ae2b07c9b4bb2a06641ba79ae478b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:30.814296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.815797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-002a0513-ac9e-4db5-95ab-2d25261aaa08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:30.822295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 976/1948] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:30 2022] DELETE /identity/v3/projects/c37ae2b07c9b4bb2a06641ba79ae478b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:30.827647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113cca4-a65c-483d-ada8-d9cc8d09b748 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:30.828388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113cca4-a65c-483d-ada8-d9cc8d09b748 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:30.829045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113cca4-a65c-483d-ada8-d9cc8d09b748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:30.866039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a113cca4-a65c-483d-ada8-d9cc8d09b748 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:30.868319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a113cca4-a65c-483d-ada8-d9cc8d09b748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.024647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 973/1949] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:31.033486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d6ddc-ebeb-4ec9-87d6-ea05f12f28a1 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.033917 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d6ddc-ebeb-4ec9-87d6-ea05f12f28a1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.034277 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d6ddc-ebeb-4ec9-87d6-ea05f12f28a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.074960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-605d6ddc-ebeb-4ec9-87d6-ea05f12f28a1 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:31.074960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-605d6ddc-ebeb-4ec9-87d6-ea05f12f28a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.097636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 977/1950] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:31.101810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113cca4-a65c-483d-ada8-d9cc8d09b748 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.120357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113cca4-a65c-483d-ada8-d9cc8d09b748 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.128315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71853fcf-5f06-49ad-9510-65496b60b216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.128695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71853fcf-5f06-49ad-9510-65496b60b216 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.129056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71853fcf-5f06-49ad-9510-65496b60b216 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.129383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71853fcf-5f06-49ad-9510-65496b60b216 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.130317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71853fcf-5f06-49ad-9510-65496b60b216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:31.131187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71853fcf-5f06-49ad-9510-65496b60b216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:31.134900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71853fcf-5f06-49ad-9510-65496b60b216 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:31.136415 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d6ddc-ebeb-4ec9-87d6-ea05f12f28a1 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.138129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-605d6ddc-ebeb-4ec9-87d6-ea05f12f28a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.155637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 974/1951] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:31.156806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71853fcf-5f06-49ad-9510-65496b60b216 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.158971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d6ddc-ebeb-4ec9-87d6-ea05f12f28a1 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.172644 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71853fcf-5f06-49ad-9510-65496b60b216 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.173718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.174869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.175134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.175400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] PATH_INFO: `/v3/projects/63dfad47e8f14958a5724e98cd30b452` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.179602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d546f-7c4a-46b4-94d6-3f7f1519ca78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.180513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d546f-7c4a-46b4-94d6-3f7f1519ca78 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.180899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d546f-7c4a-46b4-94d6-3f7f1519ca78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.181175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d546f-7c4a-46b4-94d6-3f7f1519ca78 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.182112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d546f-7c4a-46b4-94d6-3f7f1519ca78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:31.182498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=63dfad47e8f14958a5724e98cd30b452)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:31.184054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d546f-7c4a-46b4-94d6-3f7f1519ca78 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:31.186046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:31.249901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 975/1952] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:31 2022] POST /identity/v3/projects => generated 371 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:31.251895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d546f-7c4a-46b4-94d6-3f7f1519ca78 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.299596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63dfad47e8f14958a5724e98cd30b452', 'request_id': 'req-500564d6-1053-4af9-9e63-1d20e6320c6e'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.313024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63dfad47e8f14958a5724e98cd30b452', 'request_id': 'req-500564d6-1053-4af9-9e63-1d20e6320c6e'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.331000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63dfad47e8f14958a5724e98cd30b452', 'request_id': 'req-500564d6-1053-4af9-9e63-1d20e6320c6e'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.335347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:31.351511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d546f-7c4a-46b4-94d6-3f7f1519ca78 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.352812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] The token cache is being invalidate because project 63dfad47e8f14958a5724e98cd30b452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:31.353506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.355070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-500564d6-1053-4af9-9e63-1d20e6320c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.357839 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 978/1953] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:31 2022] DELETE /identity/v3/projects/63dfad47e8f14958a5724e98cd30b452 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:31.363494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f8028-b911-48d8-aad8-e7a18fcd4cdd None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.364102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f8028-b911-48d8-aad8-e7a18fcd4cdd None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.364821 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f8028-b911-48d8-aad8-e7a18fcd4cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.387823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-cf4f8028-b911-48d8-aad8-e7a18fcd4cdd None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:31.389153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-cf4f8028-b911-48d8-aad8-e7a18fcd4cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.468008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06720c-864c-418e-a596-b521a854d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.468008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06720c-864c-418e-a596-b521a854d9ec admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.468008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06720c-864c-418e-a596-b521a854d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.468008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06720c-864c-418e-a596-b521a854d9ec admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.468665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06720c-864c-418e-a596-b521a854d9ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:31.468665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06720c-864c-418e-a596-b521a854d9ec admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:31.468829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 979/1954] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:31.470663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c34185-2d5c-4398-8917-6ccb33588f9d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.471006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c34185-2d5c-4398-8917-6ccb33588f9d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.471328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c34185-2d5c-4398-8917-6ccb33588f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.503148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-33c34185-2d5c-4398-8917-6ccb33588f9d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:31.504651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-33c34185-2d5c-4398-8917-6ccb33588f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.514342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 976/1955] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:31 2022] POST /identity/v3/users => generated 286 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:31.515266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06720c-864c-418e-a596-b521a854d9ec admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.519071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-df06720c-864c-418e-a596-b521a854d9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.523733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 980/1956] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:53:31.525648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c34185-2d5c-4398-8917-6ccb33588f9d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.527783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-33c34185-2d5c-4398-8917-6ccb33588f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.543913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c34185-2d5c-4398-8917-6ccb33588f9d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.550408 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06720c-864c-418e-a596-b521a854d9ec admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.558252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb35f2d-4981-4c42-b421-53b648a4382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.559252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb35f2d-4981-4c42-b421-53b648a4382e admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.559634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb35f2d-4981-4c42-b421-53b648a4382e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.560102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb35f2d-4981-4c42-b421-53b648a4382e admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.560766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.561728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb35f2d-4981-4c42-b421-53b648a4382e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:31.562347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.562497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.562771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] PATH_INFO: `/v3/projects/87bc86ed11294134b70f5135ad7418e2` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.564328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb35f2d-4981-4c42-b421-53b648a4382e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:31.564522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] RBAC: Authorizing `identity:delete_project(project_id=87bc86ed11294134b70f5135ad7418e2)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:31.567803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:31.578354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 981/1957] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:31.579454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb35f2d-4981-4c42-b421-53b648a4382e admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.609853 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb35f2d-4981-4c42-b421-53b648a4382e admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.628591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31971aae-fa95-4c7d-9979-d582b5b4f681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.629869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31971aae-fa95-4c7d-9979-d582b5b4f681 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.630282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31971aae-fa95-4c7d-9979-d582b5b4f681 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.630687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31971aae-fa95-4c7d-9979-d582b5b4f681 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.631446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31971aae-fa95-4c7d-9979-d582b5b4f681 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:31.631759 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31971aae-fa95-4c7d-9979-d582b5b4f681 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:31.633635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31971aae-fa95-4c7d-9979-d582b5b4f681 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:31.651443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 982/1958] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:31.652776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31971aae-fa95-4c7d-9979-d582b5b4f681 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.654416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87bc86ed11294134b70f5135ad7418e2', 'request_id': 'req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.665851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31971aae-fa95-4c7d-9979-d582b5b4f681 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.675680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87bc86ed11294134b70f5135ad7418e2', 'request_id': 'req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.685613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87bc86ed11294134b70f5135ad7418e2', 'request_id': 'req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.700358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:31.707284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] The token cache is being invalidate because project 87bc86ed11294134b70f5135ad7418e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:31.707576 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.708308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.711190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 977/1959] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:31 2022] DELETE /identity/v3/projects/87bc86ed11294134b70f5135ad7418e2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:31.712637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.714280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.757148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b31f07-33ba-45a0-8d6f-73911bdeb826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.758021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b31f07-33ba-45a0-8d6f-73911bdeb826 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.758636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b31f07-33ba-45a0-8d6f-73911bdeb826 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.758991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b31f07-33ba-45a0-8d6f-73911bdeb826 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.759805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b31f07-33ba-45a0-8d6f-73911bdeb826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:31.762234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b31f07-33ba-45a0-8d6f-73911bdeb826 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:31.772360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 983/1960] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:31.773245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b31f07-33ba-45a0-8d6f-73911bdeb826 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.775774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-33b31f07-33ba-45a0-8d6f-73911bdeb826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.788625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b31f07-33ba-45a0-8d6f-73911bdeb826 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.794107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.794978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.795648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57af7-c34e-4d21-b0b6-75f0f6e4982c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.796327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.796618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] PATH_INFO: `/v3/users/2768bd566ed04158b94cde86136a6c09` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.797974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2768bd566ed04158b94cde86136a6c09)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:31.800004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:31.800323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae324689-7972-4934-8b25-2b70fa42b1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.800974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae324689-7972-4934-8b25-2b70fa42b1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.801257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae324689-7972-4934-8b25-2b70fa42b1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.801526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae324689-7972-4934-8b25-2b70fa42b1b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:31.802273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae324689-7972-4934-8b25-2b70fa42b1b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:31.804422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae324689-7972-4934-8b25-2b70fa42b1b4 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:31.818947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:31.833951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 978/1961] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:31 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:31.835128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae324689-7972-4934-8b25-2b70fa42b1b4 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.839162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ae324689-7972-4934-8b25-2b70fa42b1b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.857053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2768bd566ed04158b94cde86136a6c09'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.864538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2768bd566ed04158b94cde86136a6c09'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.865181 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2768bd566ed04158b94cde86136a6c09'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.878958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2768bd566ed04158b94cde86136a6c09'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.879690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2768bd566ed04158b94cde86136a6c09'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:31.894679 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 984/1962] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:31 2022] DELETE /identity/v3/users/2768bd566ed04158b94cde86136a6c09 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:31.897687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:31.897687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:31.911625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae324689-7972-4934-8b25-2b70fa42b1b4 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:31.987526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55ae3b-0c32-4cfa-bdd1-89e0fa3cc574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:31.989879 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55ae3b-0c32-4cfa-bdd1-89e0fa3cc574 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:31.990285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55ae3b-0c32-4cfa-bdd1-89e0fa3cc574 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:31.992598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55ae3b-0c32-4cfa-bdd1-89e0fa3cc574 admin admin] PATH_INFO: `/v3/projects/66aedf07dfbd465a9cdd7ea8747eceed/users/526ed5de4667427aa990107d654c7dda/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.011894 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc0f61-6092-4a44-b1b8-f9a134187c6f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:32.017511 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608669a-3268-455d-9bb6-569a97dd19a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:32.018312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608669a-3268-455d-9bb6-569a97dd19a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.019041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608669a-3268-455d-9bb6-569a97dd19a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.020043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608669a-3268-455d-9bb6-569a97dd19a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.021868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608669a-3268-455d-9bb6-569a97dd19a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:32.025048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608669a-3268-455d-9bb6-569a97dd19a7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:32.027847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55ae3b-0c32-4cfa-bdd1-89e0fa3cc574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66aedf07dfbd465a9cdd7ea8747eceed, user_id=526ed5de4667427aa990107d654c7dda, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:32.030567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55ae3b-0c32-4cfa-bdd1-89e0fa3cc574 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:32.055285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 985/1963] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:31 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:32.058051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 979/1964] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:31 2022] PUT /identity/v3/projects/66aedf07dfbd465a9cdd7ea8747eceed/users/526ed5de4667427aa990107d654c7dda/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:32.067193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608669a-3268-455d-9bb6-569a97dd19a7 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:32.086958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee6aee-34f0-420a-a545-80f237fd04f3 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.087240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee6aee-34f0-420a-a545-80f237fd04f3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.087542 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee6aee-34f0-420a-a545-80f237fd04f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.108995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608669a-3268-455d-9bb6-569a97dd19a7 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:32.142628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:32.153497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-84ee6aee-34f0-420a-a545-80f237fd04f3 None None] MFA Rules not processed for user `526ed5de4667427aa990107d654c7dda`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:32.156510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-84ee6aee-34f0-420a-a545-80f237fd04f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.198551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf17f9f-a64f-437b-bc8b-a261963ca174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:32.199733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf17f9f-a64f-437b-bc8b-a261963ca174 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.200371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf17f9f-a64f-437b-bc8b-a261963ca174 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.200840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf17f9f-a64f-437b-bc8b-a261963ca174 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.201810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf17f9f-a64f-437b-bc8b-a261963ca174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:32.203967 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf17f9f-a64f-437b-bc8b-a261963ca174 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:32.219042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 986/1965] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:32.223728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf17f9f-a64f-437b-bc8b-a261963ca174 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:32.237054 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf17f9f-a64f-437b-bc8b-a261963ca174 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:32.262009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e7c01-e558-48fa-b78c-3e4afa680a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:32.262668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e7c01-e558-48fa-b78c-3e4afa680a3c admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.262981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e7c01-e558-48fa-b78c-3e4afa680a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.263248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e7c01-e558-48fa-b78c-3e4afa680a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.263976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e7c01-e558-48fa-b78c-3e4afa680a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:32.265753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e7c01-e558-48fa-b78c-3e4afa680a3c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:32.283015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 987/1966] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:32.283015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e7c01-e558-48fa-b78c-3e4afa680a3c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:32.285119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-390e7c01-e558-48fa-b78c-3e4afa680a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.302928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e7c01-e558-48fa-b78c-3e4afa680a3c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:32.308621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:32.309906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.310203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.310591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] PATH_INFO: `/v3/projects/9df8a071f321415ab8a8e66e354f64b6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.312639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df8a071f321415ab8a8e66e354f64b6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:32.314044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:32.388068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 980/1967] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:32.388068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee6aee-34f0-420a-a545-80f237fd04f3 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:32.389553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df8a071f321415ab8a8e66e354f64b6', 'request_id': 'req-b010dd61-44a8-4dfe-a139-7560d228c4d3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:32.403285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df8a071f321415ab8a8e66e354f64b6', 'request_id': 'req-b010dd61-44a8-4dfe-a139-7560d228c4d3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:32.415291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df8a071f321415ab8a8e66e354f64b6', 'request_id': 'req-b010dd61-44a8-4dfe-a139-7560d228c4d3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:32.416304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee6aee-34f0-420a-a545-80f237fd04f3 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:32.432771 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] The token cache is being invalidate because project 9df8a071f321415ab8a8e66e354f64b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:32.437761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:32.438695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:32.440731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 988/1968] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:32 2022] DELETE /identity/v3/projects/9df8a071f321415ab8a8e66e354f64b6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:32.441669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:32.443632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.451750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:32.538008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311f9b0-8dd5-4aaa-85f0-4a1972a4e080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:32.538915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311f9b0-8dd5-4aaa-85f0-4a1972a4e080 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.539199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311f9b0-8dd5-4aaa-85f0-4a1972a4e080 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.539637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311f9b0-8dd5-4aaa-85f0-4a1972a4e080 admin admin] PATH_INFO: `/v3/projects/d2f75c420b714aaf8c08e510e195a56a/users/e8c56692117c4f979cb515193396e35c/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.552902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010dd61-44a8-4dfe-a139-7560d228c4d3 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:32.557991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a9fee-209d-4b43-add4-f8a97f07a512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:32.558545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311f9b0-8dd5-4aaa-85f0-4a1972a4e080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f75c420b714aaf8c08e510e195a56a, user_id=e8c56692117c4f979cb515193396e35c, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:32.558867 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a9fee-209d-4b43-add4-f8a97f07a512 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.559242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a9fee-209d-4b43-add4-f8a97f07a512 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.559613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a9fee-209d-4b43-add4-f8a97f07a512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.561358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311f9b0-8dd5-4aaa-85f0-4a1972a4e080 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:32.561658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-708a9fee-209d-4b43-add4-f8a97f07a512 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.579546 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 981/1969] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:32 2022] PUT /identity/v3/projects/d2f75c420b714aaf8c08e510e195a56a/users/e8c56692117c4f979cb515193396e35c/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:32.591632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4756f43-0e5f-4eff-bd75-ca145a5a4bc8 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.591993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4756f43-0e5f-4eff-bd75-ca145a5a4bc8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.592480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4756f43-0e5f-4eff-bd75-ca145a5a4bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.612278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d4756f43-0e5f-4eff-bd75-ca145a5a4bc8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:32.613494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d4756f43-0e5f-4eff-bd75-ca145a5a4bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.621819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:32.644194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a9fee-209d-4b43-add4-f8a97f07a512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:32.645821 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a9fee-209d-4b43-add4-f8a97f07a512 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:32.659333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 989/1970] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:32.660709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292d91c-f4d3-4407-a6ef-f1c6986f07f1 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.661003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292d91c-f4d3-4407-a6ef-f1c6986f07f1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.661275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292d91c-f4d3-4407-a6ef-f1c6986f07f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.690134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f292d91c-f4d3-4407-a6ef-f1c6986f07f1 None None] MFA Rules not processed for user `e8c56692117c4f979cb515193396e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:32.691516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f292d91c-f4d3-4407-a6ef-f1c6986f07f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.705802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 982/1971] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:32.705802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4756f43-0e5f-4eff-bd75-ca145a5a4bc8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:32.708326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d4756f43-0e5f-4eff-bd75-ca145a5a4bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.722651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4756f43-0e5f-4eff-bd75-ca145a5a4bc8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:32.728435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:32.729332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.729643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.729925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] PATH_INFO: `/v3/users/526ed5de4667427aa990107d654c7dda` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.731840 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=526ed5de4667427aa990107d654c7dda)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:32.734029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:32.788209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 990/1972] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:32.791574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeef6b-ddae-495d-9b95-342eae25ef5b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.792580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeef6b-ddae-495d-9b95-342eae25ef5b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.793360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeef6b-ddae-495d-9b95-342eae25ef5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.813288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '526ed5de4667427aa990107d654c7dda'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:32.817239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5adeef6b-ddae-495d-9b95-342eae25ef5b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:32.819231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5adeef6b-ddae-495d-9b95-342eae25ef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.830371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '526ed5de4667427aa990107d654c7dda'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:32.831205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 991/1973] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:32.832285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adeef6b-ddae-495d-9b95-342eae25ef5b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:32.835204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5adeef6b-ddae-495d-9b95-342eae25ef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.847652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '526ed5de4667427aa990107d654c7dda'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:32.852942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adeef6b-ddae-495d-9b95-342eae25ef5b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:32.859147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f200a3-43ed-4f1a-90c1-8cf2091a18b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:32.859836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f200a3-43ed-4f1a-90c1-8cf2091a18b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.860115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f200a3-43ed-4f1a-90c1-8cf2091a18b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.860396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f200a3-43ed-4f1a-90c1-8cf2091a18b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.861159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f200a3-43ed-4f1a-90c1-8cf2091a18b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:32.862346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '526ed5de4667427aa990107d654c7dda'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:32.863290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '526ed5de4667427aa990107d654c7dda'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:32.863401 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f200a3-43ed-4f1a-90c1-8cf2091a18b8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:32.879100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 983/1974] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:32 2022] DELETE /identity/v3/users/526ed5de4667427aa990107d654c7dda => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:32.879100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:32.880685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.902137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:32.916417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 992/1975] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:32 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:32.923566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208ad9d-3844-4815-83a6-8f752e5ab613 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:32.923871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208ad9d-3844-4815-83a6-8f752e5ab613 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:32.924139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208ad9d-3844-4815-83a6-8f752e5ab613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:32.966001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7208ad9d-3844-4815-83a6-8f752e5ab613 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:32.967611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7208ad9d-3844-4815-83a6-8f752e5ab613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:32.985816 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d62d1-8d18-40fc-aa42-4ddf4d58a262 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.052184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 993/1976] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:33.052184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208ad9d-3844-4815-83a6-8f752e5ab613 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.053524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7208ad9d-3844-4815-83a6-8f752e5ab613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:33.068284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208ad9d-3844-4815-83a6-8f752e5ab613 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.077161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5911da9-bb70-483b-b435-b16bf795ef52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.078371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5911da9-bb70-483b-b435-b16bf795ef52 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.078678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5911da9-bb70-483b-b435-b16bf795ef52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.078943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5911da9-bb70-483b-b435-b16bf795ef52 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.079740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5911da9-bb70-483b-b435-b16bf795ef52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.082009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5911da9-bb70-483b-b435-b16bf795ef52 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.086962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4362e8-a6eb-420a-9995-8c13fac62038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.087504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4362e8-a6eb-420a-9995-8c13fac62038 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.087819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4362e8-a6eb-420a-9995-8c13fac62038 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.088100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4362e8-a6eb-420a-9995-8c13fac62038 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.093810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4362e8-a6eb-420a-9995-8c13fac62038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:33.094261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4362e8-a6eb-420a-9995-8c13fac62038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.096365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4362e8-a6eb-420a-9995-8c13fac62038 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.108062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 994/1977] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:33 2022] POST /identity/v3/users => generated 294 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:33.108062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5911da9-bb70-483b-b435-b16bf795ef52 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.109817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b5911da9-bb70-483b-b435-b16bf795ef52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:33.113738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 984/1978] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:33.114654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4362e8-a6eb-420a-9995-8c13fac62038 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.116432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6a4362e8-a6eb-420a-9995-8c13fac62038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:33.128208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5911da9-bb70-483b-b435-b16bf795ef52 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.139491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.140436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.140615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.140870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] PATH_INFO: `/v3/users/30f1601a2cb4455393fe4f981858857d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.142249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] RBAC: Authorizing `identity:delete_user(user_id=30f1601a2cb4455393fe4f981858857d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.144305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.171060 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:33.193406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4362e8-a6eb-420a-9995-8c13fac62038 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.197981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453882c-e38f-4dcb-a36d-6bf0b98d2549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.198626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453882c-e38f-4dcb-a36d-6bf0b98d2549 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.198905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453882c-e38f-4dcb-a36d-6bf0b98d2549 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.199177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453882c-e38f-4dcb-a36d-6bf0b98d2549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.208173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30f1601a2cb4455393fe4f981858857d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:33.208931 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453882c-e38f-4dcb-a36d-6bf0b98d2549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.210404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453882c-e38f-4dcb-a36d-6bf0b98d2549 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.218900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30f1601a2cb4455393fe4f981858857d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:33.220070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30f1601a2cb4455393fe4f981858857d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:33.222302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 985/1979] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:33.228514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453882c-e38f-4dcb-a36d-6bf0b98d2549 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.252519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30f1601a2cb4455393fe4f981858857d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:33.253822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30f1601a2cb4455393fe4f981858857d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:33.270515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453882c-e38f-4dcb-a36d-6bf0b98d2549 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.276769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497266b-ab69-4561-b502-70ad91f4a0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.277798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497266b-ab69-4561-b502-70ad91f4a0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.278149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497266b-ab69-4561-b502-70ad91f4a0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.278493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497266b-ab69-4561-b502-70ad91f4a0e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.279415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497266b-ab69-4561-b502-70ad91f4a0e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.281034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497266b-ab69-4561-b502-70ad91f4a0e3 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.289197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 995/1980] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:33 2022] DELETE /identity/v3/users/30f1601a2cb4455393fe4f981858857d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:33.291418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.292940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 986/1981] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:33.295396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337e272-9cc3-4bd5-99b5-7376f52b4bff None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.295396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337e272-9cc3-4bd5-99b5-7376f52b4bff None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.295396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337e272-9cc3-4bd5-99b5-7376f52b4bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.297102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:33.317894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9337e272-9cc3-4bd5-99b5-7376f52b4bff None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:33.319460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9337e272-9cc3-4bd5-99b5-7376f52b4bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:33.386658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:33.412878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0678e71-bda6-45c6-8c07-244dfba9adee admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.417248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 987/1982] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:33.418330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337e272-9cc3-4bd5-99b5-7376f52b4bff None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.419592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de4018-5dce-40ae-9dc7-7a9b9e463d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.419858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9337e272-9cc3-4bd5-99b5-7376f52b4bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:33.420626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4018-5dce-40ae-9dc7-7a9b9e463d0f admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.421100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4018-5dce-40ae-9dc7-7a9b9e463d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.421554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4018-5dce-40ae-9dc7-7a9b9e463d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.422481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de4018-5dce-40ae-9dc7-7a9b9e463d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.424995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de4018-5dce-40ae-9dc7-7a9b9e463d0f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.436552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337e272-9cc3-4bd5-99b5-7376f52b4bff None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.443507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce17bb-900c-4cf7-88c1-7a4f341deae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.447793 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce17bb-900c-4cf7-88c1-7a4f341deae8 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.448531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce17bb-900c-4cf7-88c1-7a4f341deae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.449097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce17bb-900c-4cf7-88c1-7a4f341deae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.450375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce17bb-900c-4cf7-88c1-7a4f341deae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.454687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce17bb-900c-4cf7-88c1-7a4f341deae8 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.467846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 996/1983] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:33 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:33.469219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de4018-5dce-40ae-9dc7-7a9b9e463d0f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.471245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a4de4018-5dce-40ae-9dc7-7a9b9e463d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:33.480613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 988/1984] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:33.488111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce17bb-900c-4cf7-88c1-7a4f341deae8 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.513463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce17bb-900c-4cf7-88c1-7a4f341deae8 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.579071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de4018-5dce-40ae-9dc7-7a9b9e463d0f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.585174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a329227-a48b-4b6a-a6b9-6dec2f7ffc5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.587386 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a329227-a48b-4b6a-a6b9-6dec2f7ffc5a service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.587767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a329227-a48b-4b6a-a6b9-6dec2f7ffc5a service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.588046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a329227-a48b-4b6a-a6b9-6dec2f7ffc5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.593522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35387b5a-3e5d-490c-8c29-d29e31e97eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.594969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35387b5a-3e5d-490c-8c29-d29e31e97eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.595777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35387b5a-3e5d-490c-8c29-d29e31e97eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.596447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35387b5a-3e5d-490c-8c29-d29e31e97eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.598272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35387b5a-3e5d-490c-8c29-d29e31e97eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.604144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35387b5a-3e5d-490c-8c29-d29e31e97eb2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.607061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a329227-a48b-4b6a-a6b9-6dec2f7ffc5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.609660 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a329227-a48b-4b6a-a6b9-6dec2f7ffc5a service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.623423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 997/1985] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:33.624489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a329227-a48b-4b6a-a6b9-6dec2f7ffc5a service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.634900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a329227-a48b-4b6a-a6b9-6dec2f7ffc5a service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.641971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d069737-f0e8-457f-9b98-893031377339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.643139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d069737-f0e8-457f-9b98-893031377339 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.643492 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d069737-f0e8-457f-9b98-893031377339 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.643831 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d069737-f0e8-457f-9b98-893031377339 admin admin] PATH_INFO: `/v3/projects/842ce4c313ba4d739a1129fd54f1ca0a/users/a7398493bcc94d689f652c58c1539b52/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.647546 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 989/1986] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:33 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:33.651297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35387b5a-3e5d-490c-8c29-d29e31e97eb2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.663077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d069737-f0e8-457f-9b98-893031377339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=842ce4c313ba4d739a1129fd54f1ca0a, user_id=a7398493bcc94d689f652c58c1539b52, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.666505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d069737-f0e8-457f-9b98-893031377339 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.674456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35387b5a-3e5d-490c-8c29-d29e31e97eb2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.681675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb3d68-fe0b-479c-a4c3-3f752881e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.682714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb3d68-fe0b-479c-a4c3-3f752881e9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.683062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb3d68-fe0b-479c-a4c3-3f752881e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.683439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb3d68-fe0b-479c-a4c3-3f752881e9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.684334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb3d68-fe0b-479c-a4c3-3f752881e9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.686239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 998/1987] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:33 2022] PUT /identity/v3/projects/842ce4c313ba4d739a1129fd54f1ca0a/users/a7398493bcc94d689f652c58c1539b52/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:33.686409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb3d68-fe0b-479c-a4c3-3f752881e9a2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.701984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 990/1988] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:33.702938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d069737-f0e8-457f-9b98-893031377339 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.707433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462b763-ad13-4057-b03a-75701ae17657 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.707835 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462b763-ad13-4057-b03a-75701ae17657 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.708218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462b763-ad13-4057-b03a-75701ae17657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.711016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:33.747237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c462b763-ad13-4057-b03a-75701ae17657 None None] MFA Rules not processed for user `a7398493bcc94d689f652c58c1539b52`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:33.748396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c462b763-ad13-4057-b03a-75701ae17657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:33.835297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 991/1989] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:33.837054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462b763-ad13-4057-b03a-75701ae17657 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.891211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d069737-f0e8-457f-9b98-893031377339 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.907126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7471c6e-bd8f-4aa7-b710-c50635b15aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.908178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7471c6e-bd8f-4aa7-b710-c50635b15aad admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.908178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7471c6e-bd8f-4aa7-b710-c50635b15aad admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.908480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7471c6e-bd8f-4aa7-b710-c50635b15aad admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.909316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7471c6e-bd8f-4aa7-b710-c50635b15aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.911946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7471c6e-bd8f-4aa7-b710-c50635b15aad admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.913886 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:33.928440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 999/1990] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:33.929781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7471c6e-bd8f-4aa7-b710-c50635b15aad admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:33.943786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462b763-ad13-4057-b03a-75701ae17657 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.945445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7471c6e-bd8f-4aa7-b710-c50635b15aad admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:33.950249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf292cd-cc48-49fb-801b-c04f2fe3b50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.952054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf292cd-cc48-49fb-801b-c04f2fe3b50a admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.952589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf292cd-cc48-49fb-801b-c04f2fe3b50a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.953048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c960b01-d712-49a1-985a-5a99bc5b9b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:33.953379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf292cd-cc48-49fb-801b-c04f2fe3b50a admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.954398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c960b01-d712-49a1-985a-5a99bc5b9b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:33.954714 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf292cd-cc48-49fb-801b-c04f2fe3b50a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.954820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c960b01-d712-49a1-985a-5a99bc5b9b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:33.955099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c960b01-d712-49a1-985a-5a99bc5b9b9d admin admin] PATH_INFO: `/v3/projects/387f55f94c784e988f8c0e240f2dc1c6/users/e8bbb1244dca43f8947835d01b69da0b/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:33.957827 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf292cd-cc48-49fb-801b-c04f2fe3b50a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.981284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c960b01-d712-49a1-985a-5a99bc5b9b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=387f55f94c784e988f8c0e240f2dc1c6, user_id=e8bbb1244dca43f8947835d01b69da0b, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:33.984808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c960b01-d712-49a1-985a-5a99bc5b9b9d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:33.997621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 992/1991] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:33 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:53:34.000465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf292cd-cc48-49fb-801b-c04f2fe3b50a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:34.008581 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1000/1992] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:33 2022] PUT /identity/v3/projects/387f55f94c784e988f8c0e240f2dc1c6/users/e8bbb1244dca43f8947835d01b69da0b/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:34.015337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d5b17-506a-4a21-842a-6e0238a3f988 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.015622 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d5b17-506a-4a21-842a-6e0238a3f988 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.015901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d5b17-506a-4a21-842a-6e0238a3f988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.035346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-399d5b17-506a-4a21-842a-6e0238a3f988 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:34.036749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-399d5b17-506a-4a21-842a-6e0238a3f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:34.090907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf292cd-cc48-49fb-801b-c04f2fe3b50a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:34.115244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1001/1993] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:34.116188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d5b17-506a-4a21-842a-6e0238a3f988 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:34.117808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-399d5b17-506a-4a21-842a-6e0238a3f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:34.131030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d5b17-506a-4a21-842a-6e0238a3f988 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:34.138165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:34.139047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.139326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.139596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] PATH_INFO: `/v3/projects/0c017416f9d24837be210629a4bd94cd` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.141274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c017416f9d24837be210629a4bd94cd)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:34.143110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:34.225249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8234b3-355e-4d48-8cee-ce72d095c10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:34.226017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8234b3-355e-4d48-8cee-ce72d095c10e admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.226301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8234b3-355e-4d48-8cee-ce72d095c10e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.226577 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8234b3-355e-4d48-8cee-ce72d095c10e admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.227552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8234b3-355e-4d48-8cee-ce72d095c10e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:34.229548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8234b3-355e-4d48-8cee-ce72d095c10e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:34.260184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c017416f9d24837be210629a4bd94cd', 'request_id': 'req-578b9e1e-4799-424f-b79c-ce1d15ecb572'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.272919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c017416f9d24837be210629a4bd94cd', 'request_id': 'req-578b9e1e-4799-424f-b79c-ce1d15ecb572'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.277980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 993/1994] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:33 2022] POST /identity/v3/users => generated 300 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:34.280124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8234b3-355e-4d48-8cee-ce72d095c10e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:34.290240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c017416f9d24837be210629a4bd94cd', 'request_id': 'req-578b9e1e-4799-424f-b79c-ce1d15ecb572'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.306555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:34.316359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] The token cache is being invalidate because project 0c017416f9d24837be210629a4bd94cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:34.316911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.318479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.321200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1002/1995] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:34 2022] DELETE /identity/v3/projects/0c017416f9d24837be210629a4bd94cd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:34.327602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:34.330981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:34.380035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8234b3-355e-4d48-8cee-ce72d095c10e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:34.404133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:34.444690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b9e1e-4799-424f-b79c-ce1d15ecb572 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:34.446977 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828320f8-3116-4dec-a77d-3954f4173f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:34.447711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828320f8-3116-4dec-a77d-3954f4173f5a service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.447939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828320f8-3116-4dec-a77d-3954f4173f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.448206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828320f8-3116-4dec-a77d-3954f4173f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.450110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-828320f8-3116-4dec-a77d-3954f4173f5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:34.451686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5ebed-1929-47bd-9fea-adaa3bd6cf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:34.452393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5ebed-1929-47bd-9fea-adaa3bd6cf26 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.452677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5ebed-1929-47bd-9fea-adaa3bd6cf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.452955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5ebed-1929-47bd-9fea-adaa3bd6cf26 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.454401 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5ebed-1929-47bd-9fea-adaa3bd6cf26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:34.455317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5ebed-1929-47bd-9fea-adaa3bd6cf26 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:34.465754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828320f8-3116-4dec-a77d-3954f4173f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:34.470273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1003/1996] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:34.470423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828320f8-3116-4dec-a77d-3954f4173f5a service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:34.470528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5ebed-1929-47bd-9fea-adaa3bd6cf26 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:34.472059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-aeb5ebed-1929-47bd-9fea-adaa3bd6cf26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:34.478637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 994/1997] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:34.486316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5ebed-1929-47bd-9fea-adaa3bd6cf26 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:34.487183 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828320f8-3116-4dec-a77d-3954f4173f5a service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:34.489484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-828320f8-3116-4dec-a77d-3954f4173f5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:34.491529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:34.493079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.493605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.493991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] PATH_INFO: `/v3/projects/66aedf07dfbd465a9cdd7ea8747eceed` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.495516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66aedf07dfbd465a9cdd7ea8747eceed)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:34.497543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:34.507629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828320f8-3116-4dec-a77d-3954f4173f5a service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:34.515659 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:34.517210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.518089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.518545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] PATH_INFO: `/v3/users/08f1991e453b4945b72f66643a4c3b27` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.520506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08f1991e453b4945b72f66643a4c3b27)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:34.523052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:34.583813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66aedf07dfbd465a9cdd7ea8747eceed', 'request_id': 'req-e5c4958b-0253-4aa3-9109-05e5431a35e3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.603681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66aedf07dfbd465a9cdd7ea8747eceed', 'request_id': 'req-e5c4958b-0253-4aa3-9109-05e5431a35e3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.611245 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08f1991e453b4945b72f66643a4c3b27'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.611584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66aedf07dfbd465a9cdd7ea8747eceed', 'request_id': 'req-e5c4958b-0253-4aa3-9109-05e5431a35e3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.626966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] The token cache is being invalidate because project 66aedf07dfbd465a9cdd7ea8747eceed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:34.627627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.627971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.630567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08f1991e453b4945b72f66643a4c3b27'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.631105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1004/1998] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:34 2022] DELETE /identity/v3/projects/66aedf07dfbd465a9cdd7ea8747eceed => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:34.635652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:34.636428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:34.648062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08f1991e453b4945b72f66643a4c3b27'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.667416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08f1991e453b4945b72f66643a4c3b27'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.668270 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-469faf9c-f26e-4592-94b0-d876e4234282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08f1991e453b4945b72f66643a4c3b27'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:34.677530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 995/1999] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:34 2022] DELETE /identity/v3/users/08f1991e453b4945b72f66643a4c3b27 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:34.681509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1f4ce-4924-4213-97db-32a28f2ea5b7 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.681509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1f4ce-4924-4213-97db-32a28f2ea5b7 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.681509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1f4ce-4924-4213-97db-32a28f2ea5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.703878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ecd1f4ce-4924-4213-97db-32a28f2ea5b7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:34.705558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ecd1f4ce-4924-4213-97db-32a28f2ea5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:34.728639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4958b-0253-4aa3-9109-05e5431a35e3 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:34.762469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:34.825524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d668251-c36a-4442-9b3b-9688a6b7750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:34.826464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d668251-c36a-4442-9b3b-9688a6b7750d admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.827225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d668251-c36a-4442-9b3b-9688a6b7750d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.827225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d668251-c36a-4442-9b3b-9688a6b7750d admin admin] PATH_INFO: `/v3/projects/605ad16ffca34ed083585a1e3b683989/users/c7a0edc968104324a6b7c8dcf0ddb089/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.854036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d668251-c36a-4442-9b3b-9688a6b7750d admin admin] RBAC: Authorizing `identity:create_grant(project_id=605ad16ffca34ed083585a1e3b683989, user_id=c7a0edc968104324a6b7c8dcf0ddb089, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:34.857026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 996/2000] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:34.859041 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea82e6-eed2-415c-a782-6fc65d9f74ec None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.859339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea82e6-eed2-415c-a782-6fc65d9f74ec None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.860544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d668251-c36a-4442-9b3b-9688a6b7750d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:34.860669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea82e6-eed2-415c-a782-6fc65d9f74ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.883457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ddea82e6-eed2-415c-a782-6fc65d9f74ec None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:34.884766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ddea82e6-eed2-415c-a782-6fc65d9f74ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:34.885457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1005/2001] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:34 2022] PUT /identity/v3/projects/605ad16ffca34ed083585a1e3b683989/users/c7a0edc968104324a6b7c8dcf0ddb089/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:34.890236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d668251-c36a-4442-9b3b-9688a6b7750d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:34.893261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3d668251-c36a-4442-9b3b-9688a6b7750d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:34.942741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:34.992260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d668251-c36a-4442-9b3b-9688a6b7750d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:34.997571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229a45f-9612-44ea-92c5-3e5795ac2e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:34.998378 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229a45f-9612-44ea-92c5-3e5795ac2e39 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:34.998658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229a45f-9612-44ea-92c5-3e5795ac2e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:34.998937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229a45f-9612-44ea-92c5-3e5795ac2e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:34.999681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229a45f-9612-44ea-92c5-3e5795ac2e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:35.001394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 997/2002] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:35.004513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229a45f-9612-44ea-92c5-3e5795ac2e39 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:35.006244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea82e6-eed2-415c-a782-6fc65d9f74ec None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:35.018879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1006/2003] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:35.019805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229a45f-9612-44ea-92c5-3e5795ac2e39 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:35.020962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea82e6-eed2-415c-a782-6fc65d9f74ec None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:35.029008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f6b46-8657-49c5-883f-9444322ba74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:35.030453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f6b46-8657-49c5-883f-9444322ba74d admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:35.031068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f6b46-8657-49c5-883f-9444322ba74d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:35.031563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f6b46-8657-49c5-883f-9444322ba74d admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:35.032570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f6b46-8657-49c5-883f-9444322ba74d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:35.032731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229a45f-9612-44ea-92c5-3e5795ac2e39 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:35.034275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f6b46-8657-49c5-883f-9444322ba74d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:35.037673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f6b46-8657-49c5-883f-9444322ba74d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:35.038623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0294f-cba8-48c3-9602-91ea638b09b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:35.040328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0294f-cba8-48c3-9602-91ea638b09b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:35.040673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0294f-cba8-48c3-9602-91ea638b09b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:35.040948 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0294f-cba8-48c3-9602-91ea638b09b4 admin admin] PATH_INFO: `/v3/projects/605ad16ffca34ed083585a1e3b683989/users/c7a0edc968104324a6b7c8dcf0ddb089/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:35.043511 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0294f-cba8-48c3-9602-91ea638b09b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=605ad16ffca34ed083585a1e3b683989, user_id=c7a0edc968104324a6b7c8dcf0ddb089, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:35.046335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0294f-cba8-48c3-9602-91ea638b09b4 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:35.058665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 998/2004] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:53:35.061976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1007/2005] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:35 2022] PUT /identity/v3/projects/605ad16ffca34ed083585a1e3b683989/users/c7a0edc968104324a6b7c8dcf0ddb089/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:35.070093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f6b46-8657-49c5-883f-9444322ba74d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:35.072887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf78de-cb84-4baa-a91e-5fb4a5248b8f None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:35.073212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf78de-cb84-4baa-a91e-5fb4a5248b8f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:35.073498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf78de-cb84-4baa-a91e-5fb4a5248b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:35.085550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f6b46-8657-49c5-883f-9444322ba74d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:35.104865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-b2bf78de-cb84-4baa-a91e-5fb4a5248b8f None None] MFA Rules not processed for user `c7a0edc968104324a6b7c8dcf0ddb089`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:35.106144 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b2bf78de-cb84-4baa-a91e-5fb4a5248b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:35.186845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1008/2006] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:53:35.202695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf78de-cb84-4baa-a91e-5fb4a5248b8f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:35.204146 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b2bf78de-cb84-4baa-a91e-5fb4a5248b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:35.252035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:35.311769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf78de-cb84-4baa-a91e-5fb4a5248b8f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:35.315730 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2834eb-2126-4eec-aa4a-8e532ec407a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:35.316518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2834eb-2126-4eec-aa4a-8e532ec407a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:35.317378 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2834eb-2126-4eec-aa4a-8e532ec407a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:35.317727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2834eb-2126-4eec-aa4a-8e532ec407a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:35.318606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2834eb-2126-4eec-aa4a-8e532ec407a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:35.320690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2834eb-2126-4eec-aa4a-8e532ec407a2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:35.324551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381dfc8-473d-430b-9b09-cb607dba3acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:35.325466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381dfc8-473d-430b-9b09-cb607dba3acb service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:35.325790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381dfc8-473d-430b-9b09-cb607dba3acb service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:35.326063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381dfc8-473d-430b-9b09-cb607dba3acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:35.341808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381dfc8-473d-430b-9b09-cb607dba3acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:35.343266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381dfc8-473d-430b-9b09-cb607dba3acb service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:35.364203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1009/2007] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:53:35.372420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 999/2008] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:35 2022] POST /identity/v3/projects => generated 399 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:35.377987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381dfc8-473d-430b-9b09-cb607dba3acb service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:35.389983 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381dfc8-473d-430b-9b09-cb607dba3acb service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:35.476609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa0178-2a8c-4aea-a15f-219a93242f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:35.477901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa0178-2a8c-4aea-a15f-219a93242f33 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:35.478312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa0178-2a8c-4aea-a15f-219a93242f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:35.478591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa0178-2a8c-4aea-a15f-219a93242f33 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:35.479295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa0178-2a8c-4aea-a15f-219a93242f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:35.482388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa0178-2a8c-4aea-a15f-219a93242f33 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:35.521936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1010/2009] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:35 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:35.526996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2834eb-2126-4eec-aa4a-8e532ec407a2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:35.546834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2834eb-2126-4eec-aa4a-8e532ec407a2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:35.558015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eef67-390d-4c7f-8d64-9cabb3fb5e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:35.558473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eef67-390d-4c7f-8d64-9cabb3fb5e8d admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:35.558829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eef67-390d-4c7f-8d64-9cabb3fb5e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:35.559182 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eef67-390d-4c7f-8d64-9cabb3fb5e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:35.560002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1eef67-390d-4c7f-8d64-9cabb3fb5e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:35.561890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1eef67-390d-4c7f-8d64-9cabb3fb5e8d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:35.586972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1000/2010] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:35.589205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa0178-2a8c-4aea-a15f-219a93242f33 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:35.599972 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa0178-2a8c-4aea-a15f-219a93242f33 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:35.619762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2f7df-cda8-4d16-ae14-3c974497b4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:35.620713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2f7df-cda8-4d16-ae14-3c974497b4fe admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:35.621065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2f7df-cda8-4d16-ae14-3c974497b4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:35.621459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2f7df-cda8-4d16-ae14-3c974497b4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:35.623925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2f7df-cda8-4d16-ae14-3c974497b4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:35.625926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2f7df-cda8-4d16-ae14-3c974497b4fe admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:35.647124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1011/2011] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:35.650822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eef67-390d-4c7f-8d64-9cabb3fb5e8d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:35.673911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eef67-390d-4c7f-8d64-9cabb3fb5e8d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:35.690492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89981cee-d24a-4757-8c2c-372d0f0f524a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:35.691538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89981cee-d24a-4757-8c2c-372d0f0f524a admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:35.691981 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89981cee-d24a-4757-8c2c-372d0f0f524a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:35.692265 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89981cee-d24a-4757-8c2c-372d0f0f524a admin admin] PATH_INFO: `/v3/projects/dd9583a9ae724760ade4fc1175db2082/users/40c9c16c71f44b6598b030d485525340/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:35.709825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89981cee-d24a-4757-8c2c-372d0f0f524a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9583a9ae724760ade4fc1175db2082, user_id=40c9c16c71f44b6598b030d485525340, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:35.713533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89981cee-d24a-4757-8c2c-372d0f0f524a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:35.738087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1001/2012] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:35 2022] PUT /identity/v3/projects/dd9583a9ae724760ade4fc1175db2082/users/40c9c16c71f44b6598b030d485525340/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:35.749272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f9d7c-629c-4d10-bb9d-8072076027a0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:35.749559 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f9d7c-629c-4d10-bb9d-8072076027a0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:35.749863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f9d7c-629c-4d10-bb9d-8072076027a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:35.789009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6c4f9d7c-629c-4d10-bb9d-8072076027a0 None None] MFA Rules not processed for user `40c9c16c71f44b6598b030d485525340`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:35.790400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6c4f9d7c-629c-4d10-bb9d-8072076027a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:35.838100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89981cee-d24a-4757-8c2c-372d0f0f524a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:35.839822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-89981cee-d24a-4757-8c2c-372d0f0f524a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:35.970940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1012/2013] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:36.008522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89981cee-d24a-4757-8c2c-372d0f0f524a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:36.020247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:36.021276 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:36.024934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:36.025639 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] PATH_INFO: `/v3/projects/3d6e456327d7419583b4d47a2cc94e6f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:36.030659 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d6e456327d7419583b4d47a2cc94e6f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:36.035464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:36.072075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:36.102197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f9d7c-629c-4d10-bb9d-8072076027a0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:36.116064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6c4f9d7c-629c-4d10-bb9d-8072076027a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:36.128284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d6e456327d7419583b4d47a2cc94e6f', 'request_id': 'req-fd50deb5-3669-48b2-a966-3b34af2bff8e'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:36.143125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d6e456327d7419583b4d47a2cc94e6f', 'request_id': 'req-fd50deb5-3669-48b2-a966-3b34af2bff8e'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:36.157658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d6e456327d7419583b4d47a2cc94e6f', 'request_id': 'req-fd50deb5-3669-48b2-a966-3b34af2bff8e'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:36.195203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] The token cache is being invalidate because project 3d6e456327d7419583b4d47a2cc94e6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:36.196022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:36.199957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fd50deb5-3669-48b2-a966-3b34af2bff8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:36.205992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:36.209418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1002/2014] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:35 2022] DELETE /identity/v3/projects/3d6e456327d7419583b4d47a2cc94e6f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:36.211203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508060b-c998-4bef-addf-ee8c98b609a3 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:36.211931 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508060b-c998-4bef-addf-ee8c98b609a3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:36.217828 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508060b-c998-4bef-addf-ee8c98b609a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:36.260285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f9d7c-629c-4d10-bb9d-8072076027a0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:36.270280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d508060b-c998-4bef-addf-ee8c98b609a3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:36.272677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d508060b-c998-4bef-addf-ee8c98b609a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:36.363338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa091f-8010-4207-a30d-7f9393e25e40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:36.364858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa091f-8010-4207-a30d-7f9393e25e40 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:36.365383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa091f-8010-4207-a30d-7f9393e25e40 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:36.365896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa091f-8010-4207-a30d-7f9393e25e40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:36.371306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f2aa091f-8010-4207-a30d-7f9393e25e40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:36.442634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1003/2015] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:36.451194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabadb3-ab72-45b6-a0ff-aec534247866 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:36.451387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabadb3-ab72-45b6-a0ff-aec534247866 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:36.451658 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabadb3-ab72-45b6-a0ff-aec534247866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:36.538748 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9fabadb3-ab72-45b6-a0ff-aec534247866 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:36.557848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9fabadb3-ab72-45b6-a0ff-aec534247866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:36.559627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa091f-8010-4207-a30d-7f9393e25e40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:36.561092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa091f-8010-4207-a30d-7f9393e25e40 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:36.601570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1004/2016] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:36.608488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabadb3-ab72-45b6-a0ff-aec534247866 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:36.611190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1013/2017] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:36.635267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabadb3-ab72-45b6-a0ff-aec534247866 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:36.661444 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a894b4-5fca-4266-8f25-e4cfe540b550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:36.664305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a894b4-5fca-4266-8f25-e4cfe540b550 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:36.665426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a894b4-5fca-4266-8f25-e4cfe540b550 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:36.665790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a894b4-5fca-4266-8f25-e4cfe540b550 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:36.669442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a894b4-5fca-4266-8f25-e4cfe540b550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:36.669795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a894b4-5fca-4266-8f25-e4cfe540b550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:36.678890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a894b4-5fca-4266-8f25-e4cfe540b550 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:36.712812 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1005/2018] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:36.717813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa091f-8010-4207-a30d-7f9393e25e40 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:36.739124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa091f-8010-4207-a30d-7f9393e25e40 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:36.756416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82393f5e-06ab-4983-85b8-cfa9ea1c1a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:36.757126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82393f5e-06ab-4983-85b8-cfa9ea1c1a29 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:36.757384 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82393f5e-06ab-4983-85b8-cfa9ea1c1a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:36.757714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82393f5e-06ab-4983-85b8-cfa9ea1c1a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:36.759266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82393f5e-06ab-4983-85b8-cfa9ea1c1a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:36.761302 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82393f5e-06ab-4983-85b8-cfa9ea1c1a29 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:36.764446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8628f-bc14-46ba-bcf2-7c9da1fdf69c None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:36.764736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8628f-bc14-46ba-bcf2-7c9da1fdf69c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:36.765016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8628f-bc14-46ba-bcf2-7c9da1fdf69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:36.838099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-dad8628f-bc14-46ba-bcf2-7c9da1fdf69c None None] MFA Rules not processed for user `e8bbb1244dca43f8947835d01b69da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:36.839621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-dad8628f-bc14-46ba-bcf2-7c9da1fdf69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:36.841245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1014/2019] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:36 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:36.842333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82393f5e-06ab-4983-85b8-cfa9ea1c1a29 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:36.950108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82393f5e-06ab-4983-85b8-cfa9ea1c1a29 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:36.958678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422d327-5651-4c40-bba1-e55e369fb1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:36.959763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422d327-5651-4c40-bba1-e55e369fb1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:36.960320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422d327-5651-4c40-bba1-e55e369fb1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:36.960800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422d327-5651-4c40-bba1-e55e369fb1f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:36.961922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422d327-5651-4c40-bba1-e55e369fb1f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:36.963890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422d327-5651-4c40-bba1-e55e369fb1f0 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:37.001998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1015/2020] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:36 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:37.003144 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422d327-5651-4c40-bba1-e55e369fb1f0 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:37.004689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e422d327-5651-4c40-bba1-e55e369fb1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:37.016303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:37.033611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1006/2021] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:53:37.033611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8628f-bc14-46ba-bcf2-7c9da1fdf69c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:37.058712 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8628f-bc14-46ba-bcf2-7c9da1fdf69c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:37.069489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b465d52-9680-4179-a307-1149ebcc8c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:37.070246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b465d52-9680-4179-a307-1149ebcc8c6d admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:37.071193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b465d52-9680-4179-a307-1149ebcc8c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:37.071513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b465d52-9680-4179-a307-1149ebcc8c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:37.072318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b465d52-9680-4179-a307-1149ebcc8c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:37.075983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b465d52-9680-4179-a307-1149ebcc8c6d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:37.095596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1007/2022] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:37.097822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422d327-5651-4c40-bba1-e55e369fb1f0 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:37.102463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b465d52-9680-4179-a307-1149ebcc8c6d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:37.107188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6c080-6094-4da2-8bd3-2d3d7691444b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:37.107925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6c080-6094-4da2-8bd3-2d3d7691444b service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:37.108215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6c080-6094-4da2-8bd3-2d3d7691444b service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:37.108493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6c080-6094-4da2-8bd3-2d3d7691444b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:37.111561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4bc6c080-6094-4da2-8bd3-2d3d7691444b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:37.115197 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:37.124627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b465d52-9680-4179-a307-1149ebcc8c6d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:37.131402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a5fc2-3b1a-41b0-a648-6280cbfa527c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:37.132386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a5fc2-3b1a-41b0-a648-6280cbfa527c admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:37.132975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a5fc2-3b1a-41b0-a648-6280cbfa527c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:37.133515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a5fc2-3b1a-41b0-a648-6280cbfa527c admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:37.134473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a5fc2-3b1a-41b0-a648-6280cbfa527c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:37.136294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a5fc2-3b1a-41b0-a648-6280cbfa527c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:37.147520 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1008/2023] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:37.153898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a5fc2-3b1a-41b0-a648-6280cbfa527c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:37.174503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a5fc2-3b1a-41b0-a648-6280cbfa527c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:37.198842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6c080-6094-4da2-8bd3-2d3d7691444b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:37.199964 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70abdc-9c1c-4481-9cc3-d957a15d4e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:37.200637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6c080-6094-4da2-8bd3-2d3d7691444b service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:37.202682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70abdc-9c1c-4481-9cc3-d957a15d4e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:37.203376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70abdc-9c1c-4481-9cc3-d957a15d4e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:37.204058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70abdc-9c1c-4481-9cc3-d957a15d4e5b admin admin] PATH_INFO: `/v3/projects/163a24a30c4d45c3b65a63257e441f67/users/21fcd85231964481a591575087724625/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:37.213263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1016/2024] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:37.240123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70abdc-9c1c-4481-9cc3-d957a15d4e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=163a24a30c4d45c3b65a63257e441f67, user_id=21fcd85231964481a591575087724625, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:37.247719 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70abdc-9c1c-4481-9cc3-d957a15d4e5b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:37.276854 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1009/2025] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:37 2022] PUT /identity/v3/projects/163a24a30c4d45c3b65a63257e441f67/users/21fcd85231964481a591575087724625/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:37.295071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b15d49-9c47-428d-80e6-35dd2a90b8e2 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:37.295589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b15d49-9c47-428d-80e6-35dd2a90b8e2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:37.297264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b15d49-9c47-428d-80e6-35dd2a90b8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:37.337489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f0b15d49-9c47-428d-80e6-35dd2a90b8e2 None None] MFA Rules not processed for user `21fcd85231964481a591575087724625`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:37.339518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f0b15d49-9c47-428d-80e6-35dd2a90b8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:37.447524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1017/2026] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:37.490492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70abdc-9c1c-4481-9cc3-d957a15d4e5b admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:37.492779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9d70abdc-9c1c-4481-9cc3-d957a15d4e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:37.604580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70abdc-9c1c-4481-9cc3-d957a15d4e5b admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:37.609942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ced5a-790f-4597-9040-de64a2a6aaed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:37.610763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ced5a-790f-4597-9040-de64a2a6aaed service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:37.611173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ced5a-790f-4597-9040-de64a2a6aaed service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:37.611508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ced5a-790f-4597-9040-de64a2a6aaed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:37.659825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b15d49-9c47-428d-80e6-35dd2a90b8e2 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:37.673823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b15d49-9c47-428d-80e6-35dd2a90b8e2 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:37.699702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be570ae-8f0f-4d2a-97bf-bbb46fb4d331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:37.700483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be570ae-8f0f-4d2a-97bf-bbb46fb4d331 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:37.700843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be570ae-8f0f-4d2a-97bf-bbb46fb4d331 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:37.701215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be570ae-8f0f-4d2a-97bf-bbb46fb4d331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:37.720398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be570ae-8f0f-4d2a-97bf-bbb46fb4d331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:37.721953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be570ae-8f0f-4d2a-97bf-bbb46fb4d331 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:37.733738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1018/2027] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:53:37.750395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ced5a-790f-4597-9040-de64a2a6aaed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:37.752654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ced5a-790f-4597-9040-de64a2a6aaed service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:37.761850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be570ae-8f0f-4d2a-97bf-bbb46fb4d331 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:37.779647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1010/2028] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:37.879131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:37.915752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be570ae-8f0f-4d2a-97bf-bbb46fb4d331 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:37.923851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:37.924429 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:37.924644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:37.924921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] PATH_INFO: `/v3/users/21fcd85231964481a591575087724625` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:37.926652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=21fcd85231964481a591575087724625)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:37.931380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:38.017940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21fcd85231964481a591575087724625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:38.029638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21fcd85231964481a591575087724625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:38.030694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21fcd85231964481a591575087724625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:38.055237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21fcd85231964481a591575087724625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:38.056617 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21fcd85231964481a591575087724625'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:38.093860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1019/2029] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:37 2022] DELETE /identity/v3/users/21fcd85231964481a591575087724625 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:38.096189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7dec8-401a-4bdc-a026-ec129ee56b57 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:38.096779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7dec8-401a-4bdc-a026-ec129ee56b57 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:38.096964 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7dec8-401a-4bdc-a026-ec129ee56b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:38.140465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-4ce7dec8-401a-4bdc-a026-ec129ee56b57 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:38.144210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4ce7dec8-401a-4bdc-a026-ec129ee56b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:38.302263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1011/2030] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:38.310295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:38.311936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:38.405519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2490d3-9345-4b96-8239-c403162c2c1d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:38.409214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5cb30-1468-412d-8443-29f7e0fd0c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:38.410553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5cb30-1468-412d-8443-29f7e0fd0c2e service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:38.410844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5cb30-1468-412d-8443-29f7e0fd0c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:38.411118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5cb30-1468-412d-8443-29f7e0fd0c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:38.423416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5cb30-1468-412d-8443-29f7e0fd0c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:38.424881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5cb30-1468-412d-8443-29f7e0fd0c2e service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:38.442780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1020/2031] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:39.116031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7dec8-401a-4bdc-a026-ec129ee56b57 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:39.118494 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4ce7dec8-401a-4bdc-a026-ec129ee56b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:39.142147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7dec8-401a-4bdc-a026-ec129ee56b57 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:39.148117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:39.149060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.149346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.149647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] PATH_INFO: `/v3/projects/163a24a30c4d45c3b65a63257e441f67` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.150979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=163a24a30c4d45c3b65a63257e441f67)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:39.153826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:39.183641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:39.228801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163a24a30c4d45c3b65a63257e441f67', 'request_id': 'req-3c001cf0-16a5-402a-9452-856ac7f1f749'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:39.239118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163a24a30c4d45c3b65a63257e441f67', 'request_id': 'req-3c001cf0-16a5-402a-9452-856ac7f1f749'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:39.248839 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163a24a30c4d45c3b65a63257e441f67', 'request_id': 'req-3c001cf0-16a5-402a-9452-856ac7f1f749'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:39.262903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] The token cache is being invalidate because project 163a24a30c4d45c3b65a63257e441f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:39.263434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:39.264145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3c001cf0-16a5-402a-9452-856ac7f1f749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:39.265952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1012/2032] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:39 2022] DELETE /identity/v3/projects/163a24a30c4d45c3b65a63257e441f67 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:39.271142 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47ef7f-56b3-4c6f-ac54-99f950be016a None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.272540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47ef7f-56b3-4c6f-ac54-99f950be016a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.273019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47ef7f-56b3-4c6f-ac54-99f950be016a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.293959 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-4b47ef7f-56b3-4c6f-ac54-99f950be016a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:39.295577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4b47ef7f-56b3-4c6f-ac54-99f950be016a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:39.362008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:39.383953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0d4e4-32e1-4173-bb34-3f6c053d5f33 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.384242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0d4e4-32e1-4173-bb34-3f6c053d5f33 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.384715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0d4e4-32e1-4173-bb34-3f6c053d5f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.387303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1021/2033] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:39.403337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c1439-58f6-490c-ac64-157287248348 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.403656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c1439-58f6-490c-ac64-157287248348 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.403935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c1439-58f6-490c-ac64-157287248348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.420463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-fdc0d4e4-32e1-4173-bb34-3f6c053d5f33 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:39.422168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fdc0d4e4-32e1-4173-bb34-3f6c053d5f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:39.424155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-da1c1439-58f6-490c-ac64-157287248348 None None] MFA Rules not processed for user `e8bbb1244dca43f8947835d01b69da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:39.425349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-da1c1439-58f6-490c-ac64-157287248348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:39.435239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1013/2034] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:39.435940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0d4e4-32e1-4173-bb34-3f6c053d5f33 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:39.446209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0d4e4-32e1-4173-bb34-3f6c053d5f33 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:39.450960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334472f-e582-4fd8-9f73-113267816601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:39.451618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334472f-e582-4fd8-9f73-113267816601 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.451973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334472f-e582-4fd8-9f73-113267816601 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.452378 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334472f-e582-4fd8-9f73-113267816601 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.453110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334472f-e582-4fd8-9f73-113267816601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:39.453419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334472f-e582-4fd8-9f73-113267816601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:39.455079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334472f-e582-4fd8-9f73-113267816601 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:39.467827 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1014/2035] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:53:39.471811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334472f-e582-4fd8-9f73-113267816601 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:39.482808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334472f-e582-4fd8-9f73-113267816601 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:39.484155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1022/2036] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:39.493837 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283a873-da21-4deb-a8d9-f4bbe6bb3d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:39.494729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a873-da21-4deb-a8d9-f4bbe6bb3d79 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.495194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a873-da21-4deb-a8d9-f4bbe6bb3d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.496171 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a873-da21-4deb-a8d9-f4bbe6bb3d79 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.497055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283a873-da21-4deb-a8d9-f4bbe6bb3d79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:39.499234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283a873-da21-4deb-a8d9-f4bbe6bb3d79 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:39.538881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1015/2037] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:39 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:39.543439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c1439-58f6-490c-ac64-157287248348 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:39.563724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c1439-58f6-490c-ac64-157287248348 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:39.631813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dc7cf-4f74-452b-be77-9ab19dea1f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:39.632251 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dc7cf-4f74-452b-be77-9ab19dea1f67 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.632563 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dc7cf-4f74-452b-be77-9ab19dea1f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.632926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dc7cf-4f74-452b-be77-9ab19dea1f67 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.634470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60dc7cf-4f74-452b-be77-9ab19dea1f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:39.637276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60dc7cf-4f74-452b-be77-9ab19dea1f67 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:39.667336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1023/2038] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:39 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:53:39.668562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283a873-da21-4deb-a8d9-f4bbe6bb3d79 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:39.697250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283a873-da21-4deb-a8d9-f4bbe6bb3d79 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:39.717435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b5e97-c4a9-4b1f-b39f-daf651a72d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:39.718155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b5e97-c4a9-4b1f-b39f-daf651a72d5e admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.718439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b5e97-c4a9-4b1f-b39f-daf651a72d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.718709 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b5e97-c4a9-4b1f-b39f-daf651a72d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.719443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b5e97-c4a9-4b1f-b39f-daf651a72d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:39.721048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b5e97-c4a9-4b1f-b39f-daf651a72d5e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:39.747274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1016/2039] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:39.748739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dc7cf-4f74-452b-be77-9ab19dea1f67 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:39.771864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dc7cf-4f74-452b-be77-9ab19dea1f67 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:39.779191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a592447-48d2-4d76-b830-5a89de1d04d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:39.780434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a592447-48d2-4d76-b830-5a89de1d04d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.780434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a592447-48d2-4d76-b830-5a89de1d04d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.780621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a592447-48d2-4d76-b830-5a89de1d04d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.781322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a592447-48d2-4d76-b830-5a89de1d04d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:39.783016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a592447-48d2-4d76-b830-5a89de1d04d0 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:39.794933 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1024/2040] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:39.798885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b5e97-c4a9-4b1f-b39f-daf651a72d5e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:39.810606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b5e97-c4a9-4b1f-b39f-daf651a72d5e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:39.815329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd684c4-fb8f-4454-94f4-61bf01ccef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:39.816221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd684c4-fb8f-4454-94f4-61bf01ccef9c admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.816512 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd684c4-fb8f-4454-94f4-61bf01ccef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.816789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd684c4-fb8f-4454-94f4-61bf01ccef9c admin admin] PATH_INFO: `/v3/projects/85af127f460e4b23a09aac684813e75e/users/a82fa5a1e7c04b7ca7b4553b9399d1a5/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.844473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd684c4-fb8f-4454-94f4-61bf01ccef9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85af127f460e4b23a09aac684813e75e, user_id=a82fa5a1e7c04b7ca7b4553b9399d1a5, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:39.847205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd684c4-fb8f-4454-94f4-61bf01ccef9c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:39.887915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1017/2041] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:39 2022] PUT /identity/v3/projects/85af127f460e4b23a09aac684813e75e/users/a82fa5a1e7c04b7ca7b4553b9399d1a5/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:39.898294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423dbc2-56d5-44b0-a612-9f8ed8a50cc1 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:39.898510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423dbc2-56d5-44b0-a612-9f8ed8a50cc1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:39.898795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423dbc2-56d5-44b0-a612-9f8ed8a50cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:39.928696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7423dbc2-56d5-44b0-a612-9f8ed8a50cc1 None None] MFA Rules not processed for user `a82fa5a1e7c04b7ca7b4553b9399d1a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:39.930591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7423dbc2-56d5-44b0-a612-9f8ed8a50cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:40.008315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1025/2042] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:40.019131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd684c4-fb8f-4454-94f4-61bf01ccef9c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:40.020996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0dd684c4-fb8f-4454-94f4-61bf01ccef9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:40.105194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd684c4-fb8f-4454-94f4-61bf01ccef9c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:40.112030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d85109-51af-47da-9845-e711c0b0bf17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:40.112614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d85109-51af-47da-9845-e711c0b0bf17 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:40.112899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d85109-51af-47da-9845-e711c0b0bf17 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:40.113181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d85109-51af-47da-9845-e711c0b0bf17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:40.133351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d85109-51af-47da-9845-e711c0b0bf17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:40.136260 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d85109-51af-47da-9845-e711c0b0bf17 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:40.167565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1018/2043] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:47.186732 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423dbc2-56d5-44b0-a612-9f8ed8a50cc1 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:47.198328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423dbc2-56d5-44b0-a612-9f8ed8a50cc1 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:47.202696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e3430-fab3-463c-b5ed-622dfccf3cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:47.203802 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3430-fab3-463c-b5ed-622dfccf3cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:47.204140 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3430-fab3-463c-b5ed-622dfccf3cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:47.204636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3430-fab3-463c-b5ed-622dfccf3cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:47.207968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f34e3430-fab3-463c-b5ed-622dfccf3cb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:47.226058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:47.295392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e3430-fab3-463c-b5ed-622dfccf3cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:47.298351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e3430-fab3-463c-b5ed-622dfccf3cb0 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:47.311069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1026/2044] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:47.646358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d85109-51af-47da-9845-e711c0b0bf17 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:47.650261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-58d85109-51af-47da-9845-e711c0b0bf17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:47.699004 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:47.745572 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d85109-51af-47da-9845-e711c0b0bf17 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:47.751236 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:47.752375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:47.752721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:47.753046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] PATH_INFO: `/v3/users/40c9c16c71f44b6598b030d485525340` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:47.754304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c9c16c71f44b6598b030d485525340)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:47.757093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:47.820353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c9c16c71f44b6598b030d485525340'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:47.841716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c9c16c71f44b6598b030d485525340'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:47.853689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c9c16c71f44b6598b030d485525340'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:47.864025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c9c16c71f44b6598b030d485525340'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:47.865373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c9c16c71f44b6598b030d485525340'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:47.875794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1019/2045] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:47 2022] DELETE /identity/v3/users/40c9c16c71f44b6598b030d485525340 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:47.880424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3906-a812-48c1-bbba-a711d965d9aa None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:47.880753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3906-a812-48c1-bbba-a711d965d9aa None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:47.881084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3906-a812-48c1-bbba-a711d965d9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:47.903504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ecbf3906-a812-48c1-bbba-a711d965d9aa None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:47.904907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ecbf3906-a812-48c1-bbba-a711d965d9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:47.976444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1027/2046] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:47.983969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:47.985733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:48.046803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4632918-7c28-404d-8745-51425ec73386 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:48.050565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6d9b7-f573-4c22-8af5-8c54764ba72e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:48.051233 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6d9b7-f573-4c22-8af5-8c54764ba72e service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.051531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6d9b7-f573-4c22-8af5-8c54764ba72e service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.051814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6d9b7-f573-4c22-8af5-8c54764ba72e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.061424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6d9b7-f573-4c22-8af5-8c54764ba72e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:48.062969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6d9b7-f573-4c22-8af5-8c54764ba72e service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:48.072701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1020/2047] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:48.375611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf3906-a812-48c1-bbba-a711d965d9aa None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:48.377502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ecbf3906-a812-48c1-bbba-a711d965d9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:48.393236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf3906-a812-48c1-bbba-a711d965d9aa None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:48.399640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:48.400536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.400821 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.401263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] PATH_INFO: `/v3/projects/dd9583a9ae724760ade4fc1175db2082` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.402641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd9583a9ae724760ade4fc1175db2082)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:48.404547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:48.454794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd9583a9ae724760ade4fc1175db2082', 'request_id': 'req-3214e35b-80ce-46f3-ac3c-97906a855fb3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:48.464029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd9583a9ae724760ade4fc1175db2082', 'request_id': 'req-3214e35b-80ce-46f3-ac3c-97906a855fb3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:48.470810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd9583a9ae724760ade4fc1175db2082', 'request_id': 'req-3214e35b-80ce-46f3-ac3c-97906a855fb3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:48.482580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] The token cache is being invalidate because project dd9583a9ae724760ade4fc1175db2082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:48.482862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:48.483480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3214e35b-80ce-46f3-ac3c-97906a855fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:48.485032 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1028/2048] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:48 2022] DELETE /identity/v3/projects/dd9583a9ae724760ade4fc1175db2082 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:48.491104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394462d-d9c1-4499-825f-20cff6c1d2ce None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.491391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394462d-d9c1-4499-825f-20cff6c1d2ce None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.491734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394462d-d9c1-4499-825f-20cff6c1d2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.510892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c394462d-d9c1-4499-825f-20cff6c1d2ce None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:48.512081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c394462d-d9c1-4499-825f-20cff6c1d2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:48.565021 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:48.580070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1021/2049] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:48.588729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c8f07-b22f-419a-a425-1d14924f7e6b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.589018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c8f07-b22f-419a-a425-1d14924f7e6b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.589298 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c8f07-b22f-419a-a425-1d14924f7e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.608313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-bd0c8f07-b22f-419a-a425-1d14924f7e6b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:48.609565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-bd0c8f07-b22f-419a-a425-1d14924f7e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:48.617655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1029/2050] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:48.621913 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394462d-d9c1-4499-825f-20cff6c1d2ce None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:48.631649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394462d-d9c1-4499-825f-20cff6c1d2ce None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:48.636231 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931dc96-fd2e-4c32-9f5c-8eadc90c0053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:48.636883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931dc96-fd2e-4c32-9f5c-8eadc90c0053 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.637161 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931dc96-fd2e-4c32-9f5c-8eadc90c0053 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.637432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931dc96-fd2e-4c32-9f5c-8eadc90c0053 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.638215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931dc96-fd2e-4c32-9f5c-8eadc90c0053 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:48.638513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931dc96-fd2e-4c32-9f5c-8eadc90c0053 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:48.640112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931dc96-fd2e-4c32-9f5c-8eadc90c0053 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:48.652817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1022/2051] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:48.657184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c8f07-b22f-419a-a425-1d14924f7e6b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:48.667611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c8f07-b22f-419a-a425-1d14924f7e6b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:48.672252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9bda5-897f-4554-acdf-3d4dd1218875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:48.672930 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9bda5-897f-4554-acdf-3d4dd1218875 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.673209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9bda5-897f-4554-acdf-3d4dd1218875 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.673482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9bda5-897f-4554-acdf-3d4dd1218875 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.674215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9bda5-897f-4554-acdf-3d4dd1218875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:48.676130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9bda5-897f-4554-acdf-3d4dd1218875 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:48.702458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1030/2052] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:48 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:48.706780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931dc96-fd2e-4c32-9f5c-8eadc90c0053 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:48.717532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931dc96-fd2e-4c32-9f5c-8eadc90c0053 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:48.781042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcaeb8-2acf-4d0d-8a2f-6b7a913ffee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:48.781729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcaeb8-2acf-4d0d-8a2f-6b7a913ffee6 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.782297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcaeb8-2acf-4d0d-8a2f-6b7a913ffee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.782584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bcaeb8-2acf-4d0d-8a2f-6b7a913ffee6 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.783456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bcaeb8-2acf-4d0d-8a2f-6b7a913ffee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:48.785694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bcaeb8-2acf-4d0d-8a2f-6b7a913ffee6 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:48.814559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1023/2053] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:48 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:53:48.818431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9bda5-897f-4554-acdf-3d4dd1218875 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:48.839387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9bda5-897f-4554-acdf-3d4dd1218875 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:48.844424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14034a3-1f3f-4882-a595-ee28fd48011c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:48.845263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14034a3-1f3f-4882-a595-ee28fd48011c admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.845553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14034a3-1f3f-4882-a595-ee28fd48011c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.846118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14034a3-1f3f-4882-a595-ee28fd48011c admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.846612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14034a3-1f3f-4882-a595-ee28fd48011c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:48.848349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14034a3-1f3f-4882-a595-ee28fd48011c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:48.849872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:48.862089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1031/2054] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:48.866098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcaeb8-2acf-4d0d-8a2f-6b7a913ffee6 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:48.879149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bcaeb8-2acf-4d0d-8a2f-6b7a913ffee6 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:48.885159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b8fda-84b5-4fc5-ada2-31cb1b3a4a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:48.885879 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b8fda-84b5-4fc5-ada2-31cb1b3a4a9c admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.886164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b8fda-84b5-4fc5-ada2-31cb1b3a4a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.889638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b8fda-84b5-4fc5-ada2-31cb1b3a4a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.889638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b8fda-84b5-4fc5-ada2-31cb1b3a4a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:48.889638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b8fda-84b5-4fc5-ada2-31cb1b3a4a9c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:48.901362 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1024/2055] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:48.905996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14034a3-1f3f-4882-a595-ee28fd48011c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:48.918892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14034a3-1f3f-4882-a595-ee28fd48011c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:48.924618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca0839-6cfa-42f8-b653-d4fea573081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:48.925715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca0839-6cfa-42f8-b653-d4fea573081d admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.925873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca0839-6cfa-42f8-b653-d4fea573081d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.926141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca0839-6cfa-42f8-b653-d4fea573081d admin admin] PATH_INFO: `/v3/projects/669587fe364c4741b489449451f2294c/users/c815ea64d44a48d89a2f5ff10f3f9ee8/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.943330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca0839-6cfa-42f8-b653-d4fea573081d admin admin] RBAC: Authorizing `identity:create_grant(project_id=669587fe364c4741b489449451f2294c, user_id=c815ea64d44a48d89a2f5ff10f3f9ee8, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:48.947286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca0839-6cfa-42f8-b653-d4fea573081d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:48.962028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1032/2056] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:48 2022] PUT /identity/v3/projects/669587fe364c4741b489449451f2294c/users/c815ea64d44a48d89a2f5ff10f3f9ee8/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:48.967767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9727f21-9f37-4048-80dd-23a5ce440b6f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:48.968126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9727f21-9f37-4048-80dd-23a5ce440b6f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:48.968459 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9727f21-9f37-4048-80dd-23a5ce440b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:48.988682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f9727f21-9f37-4048-80dd-23a5ce440b6f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:48.989918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f9727f21-9f37-4048-80dd-23a5ce440b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:49.052791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1025/2057] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:49.062661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca0839-6cfa-42f8-b653-d4fea573081d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:49.064513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a8ca0839-6cfa-42f8-b653-d4fea573081d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:49.123459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca0839-6cfa-42f8-b653-d4fea573081d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:49.127162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d8b37-e24f-4ebf-abe3-ad91630a5660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:49.127807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d8b37-e24f-4ebf-abe3-ad91630a5660 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:49.128112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d8b37-e24f-4ebf-abe3-ad91630a5660 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:49.128381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d8b37-e24f-4ebf-abe3-ad91630a5660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:49.138848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d8b37-e24f-4ebf-abe3-ad91630a5660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:49.140288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d8b37-e24f-4ebf-abe3-ad91630a5660 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:49.150455 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1033/2058] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:49.673567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9727f21-9f37-4048-80dd-23a5ce440b6f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:49.675753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f9727f21-9f37-4048-80dd-23a5ce440b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:49.698009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9727f21-9f37-4048-80dd-23a5ce440b6f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:49.704460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:49.705703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:49.706067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:49.706550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] PATH_INFO: `/v3/users/e8c56692117c4f979cb515193396e35c` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:49.708019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c56692117c4f979cb515193396e35c)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:49.721089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:49.797044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c56692117c4f979cb515193396e35c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:49.805112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:49.817352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c56692117c4f979cb515193396e35c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:49.830202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c56692117c4f979cb515193396e35c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:49.839052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c56692117c4f979cb515193396e35c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:49.839751 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c56692117c4f979cb515193396e35c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:49.847070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1026/2059] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:49 2022] DELETE /identity/v3/users/e8c56692117c4f979cb515193396e35c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:49.852049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2eaa7-4524-4daf-bd28-879b13a51d63 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:49.852515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2eaa7-4524-4daf-bd28-879b13a51d63 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:49.852795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2eaa7-4524-4daf-bd28-879b13a51d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:49.873827 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-0dd2eaa7-4524-4daf-bd28-879b13a51d63 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:49.875245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0dd2eaa7-4524-4daf-bd28-879b13a51d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:49.941042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1034/2060] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:53:49.951813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:49.953548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:49.975643 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2eaa7-4524-4daf-bd28-879b13a51d63 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:49.978191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0dd2eaa7-4524-4daf-bd28-879b13a51d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:49.994450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2eaa7-4524-4daf-bd28-879b13a51d63 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:50.000411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:50.002287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:50.002628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:50.002837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] PATH_INFO: `/v3/users/a82fa5a1e7c04b7ca7b4553b9399d1a5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:50.004470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a82fa5a1e7c04b7ca7b4553b9399d1a5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:50.007354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:50.018426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fe5a1-9623-4a3b-9cb8-c6431b68733e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:50.022805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb31d3a-ec94-4ca6-8b55-8eb41f8a98f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:50.024062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb31d3a-ec94-4ca6-8b55-8eb41f8a98f5 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:50.024597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb31d3a-ec94-4ca6-8b55-8eb41f8a98f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:50.025152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb31d3a-ec94-4ca6-8b55-8eb41f8a98f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:50.036593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb31d3a-ec94-4ca6-8b55-8eb41f8a98f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:50.040569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb31d3a-ec94-4ca6-8b55-8eb41f8a98f5 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:50.053477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1027/2061] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:50.070156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a82fa5a1e7c04b7ca7b4553b9399d1a5'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.082203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a82fa5a1e7c04b7ca7b4553b9399d1a5'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.083459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a82fa5a1e7c04b7ca7b4553b9399d1a5'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.100138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:50.102933 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a82fa5a1e7c04b7ca7b4553b9399d1a5'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.104476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a82fa5a1e7c04b7ca7b4553b9399d1a5'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.151316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1035/2062] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:49 2022] DELETE /identity/v3/users/a82fa5a1e7c04b7ca7b4553b9399d1a5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:50.153401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a1d5b-039e-4d45-9ad3-e374dfc7b9dc None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:50.153824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a1d5b-039e-4d45-9ad3-e374dfc7b9dc None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:50.154339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a1d5b-039e-4d45-9ad3-e374dfc7b9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:50.186366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d76a1d5b-039e-4d45-9ad3-e374dfc7b9dc None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:50.187976 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d76a1d5b-039e-4d45-9ad3-e374dfc7b9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:50.263815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1028/2063] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:50.278397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:50.281037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:50.420708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a1d5b-039e-4d45-9ad3-e374dfc7b9dc None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:50.426512 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d76a1d5b-039e-4d45-9ad3-e374dfc7b9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:50.433162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111e27f-369a-4934-856e-eed36fa5c3fb admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:50.440561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532fae-8cdf-449e-b7f1-d28e665ece7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:50.441325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532fae-8cdf-449e-b7f1-d28e665ece7d service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:50.441961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532fae-8cdf-449e-b7f1-d28e665ece7d service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:50.443433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532fae-8cdf-449e-b7f1-d28e665ece7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:50.455308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a1d5b-039e-4d45-9ad3-e374dfc7b9dc None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:50.459421 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f532fae-8cdf-449e-b7f1-d28e665ece7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:50.462757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f532fae-8cdf-449e-b7f1-d28e665ece7d service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:50.479360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:50.480280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:50.480565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:50.480832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] PATH_INFO: `/v3/projects/d2f75c420b714aaf8c08e510e195a56a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:50.490576 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1036/2064] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:50.491049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f75c420b714aaf8c08e510e195a56a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:50.501612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:50.579138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f75c420b714aaf8c08e510e195a56a', 'request_id': 'req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.592309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f75c420b714aaf8c08e510e195a56a', 'request_id': 'req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.609905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f75c420b714aaf8c08e510e195a56a', 'request_id': 'req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.620649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:50.626345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] The token cache is being invalidate because project d2f75c420b714aaf8c08e510e195a56a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:50.626901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.628425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.630672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1029/2065] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:50 2022] DELETE /identity/v3/projects/d2f75c420b714aaf8c08e510e195a56a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:50.635661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532fae-8cdf-449e-b7f1-d28e665ece7d service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:50.653203 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532fae-8cdf-449e-b7f1-d28e665ece7d service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:50.756846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:50.758548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:50.758777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:50.759057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] PATH_INFO: `/v3/users/a7398493bcc94d689f652c58c1539b52` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:50.760392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7398493bcc94d689f652c58c1539b52)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:50.762801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:50.818215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7398493bcc94d689f652c58c1539b52'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.826187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7398493bcc94d689f652c58c1539b52'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.826810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7398493bcc94d689f652c58c1539b52'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.839734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7398493bcc94d689f652c58c1539b52'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.840426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-50d5c428-18e5-4226-ad89-2ffdbd401819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7398493bcc94d689f652c58c1539b52'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:50.854746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1037/2066] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:50 2022] DELETE /identity/v3/users/a7398493bcc94d689f652c58c1539b52 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:51.127091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:51.129573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:51.146290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3989aa-cdb6-4f39-b9a1-74c01e2a0298 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:51.250815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:51.252439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:51.252750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:51.254544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] PATH_INFO: `/v3/projects/85af127f460e4b23a09aac684813e75e` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:51.256088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=85af127f460e4b23a09aac684813e75e)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:51.259112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:51.325008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85af127f460e4b23a09aac684813e75e', 'request_id': 'req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.334941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85af127f460e4b23a09aac684813e75e', 'request_id': 'req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.343073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85af127f460e4b23a09aac684813e75e', 'request_id': 'req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.365938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] The token cache is being invalidate because project 85af127f460e4b23a09aac684813e75e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:51.366073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.367322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.369385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1030/2067] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:51 2022] DELETE /identity/v3/projects/85af127f460e4b23a09aac684813e75e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:51.374631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb59075-246f-4b61-9acb-98c2599fe2f3 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:51.374920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb59075-246f-4b61-9acb-98c2599fe2f3 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:51.375191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb59075-246f-4b61-9acb-98c2599fe2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:51.398425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9bb59075-246f-4b61-9acb-98c2599fe2f3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:51.399845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9bb59075-246f-4b61-9acb-98c2599fe2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:51.404908 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:51.409104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:51.415815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:51.488344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1038/2068] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:51.490928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfc2da-6d70-41c5-83cd-fd2f3482e7dc admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:51.497631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb6abf-d76d-4702-b930-551204fe07e0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:51.497765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb6abf-d76d-4702-b930-551204fe07e0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:51.498123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb6abf-d76d-4702-b930-551204fe07e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:51.499456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:51.505346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:51.505346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:51.505346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] PATH_INFO: `/v3/projects/842ce4c313ba4d739a1129fd54f1ca0a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:51.505346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=842ce4c313ba4d739a1129fd54f1ca0a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:51.509288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:51.522606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f2eb6abf-d76d-4702-b930-551204fe07e0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:51.525271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f2eb6abf-d76d-4702-b930-551204fe07e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:51.545062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1039/2069] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:51.551899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb6abf-d76d-4702-b930-551204fe07e0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:51.563233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb6abf-d76d-4702-b930-551204fe07e0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:51.571507 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30840368-a3c2-4953-b03e-9acb5a037121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:51.573120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30840368-a3c2-4953-b03e-9acb5a037121 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:51.573120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30840368-a3c2-4953-b03e-9acb5a037121 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:51.573376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30840368-a3c2-4953-b03e-9acb5a037121 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:51.574138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30840368-a3c2-4953-b03e-9acb5a037121 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:51.574452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30840368-a3c2-4953-b03e-9acb5a037121 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:51.575188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:51.576299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30840368-a3c2-4953-b03e-9acb5a037121 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:51.590631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1040/2070] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:51.594960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30840368-a3c2-4953-b03e-9acb5a037121 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:51.606060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '842ce4c313ba4d739a1129fd54f1ca0a', 'request_id': 'req-313a97e3-6ba7-44b5-8da1-07318f814e72'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.607580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30840368-a3c2-4953-b03e-9acb5a037121 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:51.622077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '842ce4c313ba4d739a1129fd54f1ca0a', 'request_id': 'req-313a97e3-6ba7-44b5-8da1-07318f814e72'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.636645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '842ce4c313ba4d739a1129fd54f1ca0a', 'request_id': 'req-313a97e3-6ba7-44b5-8da1-07318f814e72'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.652923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] The token cache is being invalidate because project 842ce4c313ba4d739a1129fd54f1ca0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:51.653384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.655126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.661973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1031/2071] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:51 2022] DELETE /identity/v3/projects/842ce4c313ba4d739a1129fd54f1ca0a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:51.662902 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:51.666082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:51.699900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b139519-0403-490c-bffb-4f54014c7161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:51.702101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b139519-0403-490c-bffb-4f54014c7161 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:51.702538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b139519-0403-490c-bffb-4f54014c7161 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:51.702927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b139519-0403-490c-bffb-4f54014c7161 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:51.704432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b139519-0403-490c-bffb-4f54014c7161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:51.706175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b139519-0403-490c-bffb-4f54014c7161 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:51.741323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1041/2072] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:51 2022] POST /identity/v3/projects => generated 361 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:53:51.745484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b139519-0403-490c-bffb-4f54014c7161 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:51.749413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7b139519-0403-490c-bffb-4f54014c7161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:51.771925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a97e3-6ba7-44b5-8da1-07318f814e72 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:51.844437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:51.845563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:51.845906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:51.846204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] PATH_INFO: `/v3/users/c7a0edc968104324a6b7c8dcf0ddb089` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:51.847487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a0edc968104324a6b7c8dcf0ddb089)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:51.849511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:51.866317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b139519-0403-490c-bffb-4f54014c7161 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:51.871444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3215a4-f352-46b7-b192-8cca3cb1f604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:51.872414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3215a4-f352-46b7-b192-8cca3cb1f604 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:51.872762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3215a4-f352-46b7-b192-8cca3cb1f604 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:51.873273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3215a4-f352-46b7-b192-8cca3cb1f604 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:51.874741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3215a4-f352-46b7-b192-8cca3cb1f604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:51.876328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3215a4-f352-46b7-b192-8cca3cb1f604 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:51.905192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1042/2073] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:51 2022] POST /identity/v3/users => generated 281 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:51.910123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3215a4-f352-46b7-b192-8cca3cb1f604 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:51.918028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:51.922353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3215a4-f352-46b7-b192-8cca3cb1f604 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:51.924634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a0edc968104324a6b7c8dcf0ddb089'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.935260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2e338-cb1a-4b19-bdba-615b908c3674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:51.935944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2e338-cb1a-4b19-bdba-615b908c3674 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:51.936227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2e338-cb1a-4b19-bdba-615b908c3674 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:51.936502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2e338-cb1a-4b19-bdba-615b908c3674 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:51.937271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2e338-cb1a-4b19-bdba-615b908c3674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:51.939097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2e338-cb1a-4b19-bdba-615b908c3674 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:51.947133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a0edc968104324a6b7c8dcf0ddb089'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.953627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1043/2074] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:51.954357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:51.955149 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2e338-cb1a-4b19-bdba-615b908c3674 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:51.960948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a0edc968104324a6b7c8dcf0ddb089'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.967704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2e338-cb1a-4b19-bdba-615b908c3674 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:51.972458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a0edc968104324a6b7c8dcf0ddb089'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.973357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a0edc968104324a6b7c8dcf0ddb089'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:51.974116 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98589fd-dba5-4d06-a057-2ade712e7817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:51.974877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98589fd-dba5-4d06-a057-2ade712e7817 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:51.975233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98589fd-dba5-4d06-a057-2ade712e7817 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:51.975587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98589fd-dba5-4d06-a057-2ade712e7817 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:51.976756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98589fd-dba5-4d06-a057-2ade712e7817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:51.978137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98589fd-dba5-4d06-a057-2ade712e7817 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:51.980477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1032/2075] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:51 2022] DELETE /identity/v3/users/c7a0edc968104324a6b7c8dcf0ddb089 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:52.003804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1044/2076] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:52.005013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:52.006628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:52.134123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b0d03-c4ab-4187-9910-746e081f18c5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:52.159029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecafb99-8f7a-4741-a944-f5514e860424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:52.160064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecafb99-8f7a-4741-a944-f5514e860424 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:52.160826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecafb99-8f7a-4741-a944-f5514e860424 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:52.161630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecafb99-8f7a-4741-a944-f5514e860424 admin admin] PATH_INFO: `/v3/projects/5a8121c7fe714df88e59663263d4f0c2/users/46e1e3d0b55647a19eee579d30f2758a/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:52.220946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecafb99-8f7a-4741-a944-f5514e860424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8121c7fe714df88e59663263d4f0c2, user_id=46e1e3d0b55647a19eee579d30f2758a, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:52.235148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecafb99-8f7a-4741-a944-f5514e860424 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:52.256929 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1033/2077] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:52 2022] PUT /identity/v3/projects/5a8121c7fe714df88e59663263d4f0c2/users/46e1e3d0b55647a19eee579d30f2758a/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:52.268897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4d81f-de0d-4fa4-87c0-6209be9ab2f2 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:52.269747 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4d81f-de0d-4fa4-87c0-6209be9ab2f2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:52.270241 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4d81f-de0d-4fa4-87c0-6209be9ab2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:52.325476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-92b4d81f-de0d-4fa4-87c0-6209be9ab2f2 None None] MFA Rules not processed for user `46e1e3d0b55647a19eee579d30f2758a`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:52.326798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-92b4d81f-de0d-4fa4-87c0-6209be9ab2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:52.409547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1045/2078] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:52.444511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecafb99-8f7a-4741-a944-f5514e860424 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:52.447524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fecafb99-8f7a-4741-a944-f5514e860424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:52.548085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecafb99-8f7a-4741-a944-f5514e860424 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:52.556157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833a33c-19ec-4d63-b10d-1b7519d5e444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:52.556843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833a33c-19ec-4d63-b10d-1b7519d5e444 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:52.557123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833a33c-19ec-4d63-b10d-1b7519d5e444 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:52.557395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833a33c-19ec-4d63-b10d-1b7519d5e444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:52.567463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833a33c-19ec-4d63-b10d-1b7519d5e444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:52.568645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833a33c-19ec-4d63-b10d-1b7519d5e444 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:52.579089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1034/2079] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:52.596321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4d81f-de0d-4fa4-87c0-6209be9ab2f2 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:52.685316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833a33c-19ec-4d63-b10d-1b7519d5e444 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:52.687525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6833a33c-19ec-4d63-b10d-1b7519d5e444 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:52.691661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4d81f-de0d-4fa4-87c0-6209be9ab2f2 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:52.698457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:52.699488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:52.700016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:52.700397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] PATH_INFO: `/v3/users/46e1e3d0b55647a19eee579d30f2758a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:52.702327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e1e3d0b55647a19eee579d30f2758a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:52.704502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:52.713292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833a33c-19ec-4d63-b10d-1b7519d5e444 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:52.718896 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:52.720035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:52.720316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:52.720589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] PATH_INFO: `/v3/projects/605ad16ffca34ed083585a1e3b683989` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:52.721904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=605ad16ffca34ed083585a1e3b683989)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:52.723851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:52.792551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e1e3d0b55647a19eee579d30f2758a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:52.801972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '605ad16ffca34ed083585a1e3b683989', 'request_id': 'req-6f8f77b1-0fc7-460c-a489-31da842938ee'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:52.807915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e1e3d0b55647a19eee579d30f2758a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:52.808391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e1e3d0b55647a19eee579d30f2758a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:52.813789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '605ad16ffca34ed083585a1e3b683989', 'request_id': 'req-6f8f77b1-0fc7-460c-a489-31da842938ee'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:52.821499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '605ad16ffca34ed083585a1e3b683989', 'request_id': 'req-6f8f77b1-0fc7-460c-a489-31da842938ee'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:52.823382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:52.830846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e1e3d0b55647a19eee579d30f2758a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:52.831890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e95b46-e93f-4099-9ede-1fbb6dda6ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e1e3d0b55647a19eee579d30f2758a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:52.838307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:52.840214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] The token cache is being invalidate because project 605ad16ffca34ed083585a1e3b683989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:52.840794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:52.841524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6f8f77b1-0fc7-460c-a489-31da842938ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:52.843733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1035/2080] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:52 2022] DELETE /identity/v3/projects/605ad16ffca34ed083585a1e3b683989 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:52.849698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507501-f209-4bfe-a940-5f814ee3ada5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:52.849982 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507501-f209-4bfe-a940-5f814ee3ada5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:52.850258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507501-f209-4bfe-a940-5f814ee3ada5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:52.855087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1046/2081] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:53:52 2022] DELETE /identity/v3/users/46e1e3d0b55647a19eee579d30f2758a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:52.858359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d891-3aa6-47e3-806c-24d6fe60d721 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:52.858648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d891-3aa6-47e3-806c-24d6fe60d721 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:52.859151 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d891-3aa6-47e3-806c-24d6fe60d721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:52.874668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7b507501-f209-4bfe-a940-5f814ee3ada5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:52.875941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7b507501-f209-4bfe-a940-5f814ee3ada5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:52.878391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6217d891-3aa6-47e3-806c-24d6fe60d721 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:52.879598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6217d891-3aa6-47e3-806c-24d6fe60d721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:52.981009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1036/2082] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:52.987086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e4e7-982b-4c0a-a56e-4f8a95b29a52 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:52.987378 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e4e7-982b-4c0a-a56e-4f8a95b29a52 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:52.987973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e4e7-982b-4c0a-a56e-4f8a95b29a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:52.988092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1047/2083] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:52.999189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217d891-3aa6-47e3-806c-24d6fe60d721 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:53.001143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6217d891-3aa6-47e3-806c-24d6fe60d721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:53.020901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-3c12e4e7-982b-4c0a-a56e-4f8a95b29a52 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:53.035066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3c12e4e7-982b-4c0a-a56e-4f8a95b29a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:53.049598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1037/2084] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:53.053674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12e4e7-982b-4c0a-a56e-4f8a95b29a52 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:53.064536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12e4e7-982b-4c0a-a56e-4f8a95b29a52 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:53.086100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed4d57-6685-4ff7-b831-98fab7d0a3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:53.087032 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed4d57-6685-4ff7-b831-98fab7d0a3bb admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:53.087562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed4d57-6685-4ff7-b831-98fab7d0a3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:53.087988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed4d57-6685-4ff7-b831-98fab7d0a3bb admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:53.089044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed4d57-6685-4ff7-b831-98fab7d0a3bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:53.089265 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed4d57-6685-4ff7-b831-98fab7d0a3bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:53.094058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed4d57-6685-4ff7-b831-98fab7d0a3bb admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:53.114963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1038/2085] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:53:53.119809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed4d57-6685-4ff7-b831-98fab7d0a3bb admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:53.121568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217d891-3aa6-47e3-806c-24d6fe60d721 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:53.125408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60817bb-a7a4-4903-9f89-148f4f2d0663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:53.126428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60817bb-a7a4-4903-9f89-148f4f2d0663 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:53.126782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60817bb-a7a4-4903-9f89-148f4f2d0663 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:53.127063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60817bb-a7a4-4903-9f89-148f4f2d0663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:53.133136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed4d57-6685-4ff7-b831-98fab7d0a3bb admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:53.138950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996509dd-9ed0-4750-aa74-42a816f7a34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:53.139655 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996509dd-9ed0-4750-aa74-42a816f7a34c admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:53.139913 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996509dd-9ed0-4750-aa74-42a816f7a34c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:53.140193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996509dd-9ed0-4750-aa74-42a816f7a34c admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:53.140769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60817bb-a7a4-4903-9f89-148f4f2d0663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:53.140884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996509dd-9ed0-4750-aa74-42a816f7a34c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:53.143723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60817bb-a7a4-4903-9f89-148f4f2d0663 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:53.153155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996509dd-9ed0-4750-aa74-42a816f7a34c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:53.158021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1048/2086] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:53.198353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1039/2087] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:53 2022] POST /identity/v3/projects => generated 409 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:53.204785 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60817bb-a7a4-4903-9f89-148f4f2d0663 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:53.220651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60817bb-a7a4-4903-9f89-148f4f2d0663 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:53.352319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dbc06-8aa7-4d0b-b535-0d60f8296692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:53.352938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dbc06-8aa7-4d0b-b535-0d60f8296692 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:53.353219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dbc06-8aa7-4d0b-b535-0d60f8296692 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:53.353501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608dbc06-8aa7-4d0b-b535-0d60f8296692 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:53.354223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608dbc06-8aa7-4d0b-b535-0d60f8296692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:53.356248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608dbc06-8aa7-4d0b-b535-0d60f8296692 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:53.407996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1049/2088] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:53 2022] POST /identity/v3/users => generated 305 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:53.412418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996509dd-9ed0-4750-aa74-42a816f7a34c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:53.430800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996509dd-9ed0-4750-aa74-42a816f7a34c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:53.442438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8f335-e8c3-486c-a429-dbf99eae38bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:53.445259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8f335-e8c3-486c-a429-dbf99eae38bd admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:53.447076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8f335-e8c3-486c-a429-dbf99eae38bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:53.447360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8f335-e8c3-486c-a429-dbf99eae38bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:53.448136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8f335-e8c3-486c-a429-dbf99eae38bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:53.456200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8f335-e8c3-486c-a429-dbf99eae38bd admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:53.505829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1040/2089] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:53.508537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dbc06-8aa7-4d0b-b535-0d60f8296692 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:53.531216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608dbc06-8aa7-4d0b-b535-0d60f8296692 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:53.560104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3bb48-5171-4055-91d1-378acec29e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:53.561421 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3bb48-5171-4055-91d1-378acec29e97 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:53.561751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3bb48-5171-4055-91d1-378acec29e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:53.563473 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3bb48-5171-4055-91d1-378acec29e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:53.565223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3bb48-5171-4055-91d1-378acec29e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:53.572481 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3bb48-5171-4055-91d1-378acec29e97 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:53.633202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1050/2090] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:53.633827 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8f335-e8c3-486c-a429-dbf99eae38bd admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:53.657848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8f335-e8c3-486c-a429-dbf99eae38bd admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:53.675823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7b9a9-465d-4844-b112-0248aa5b2edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:53.677930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7b9a9-465d-4844-b112-0248aa5b2edd admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:53.679504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7b9a9-465d-4844-b112-0248aa5b2edd admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:53.680505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3bb48-5171-4055-91d1-378acec29e97 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:53.681036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7b9a9-465d-4844-b112-0248aa5b2edd admin admin] PATH_INFO: `/v3/projects/a572e7d81e2246768d8af3d0f70be94e/users/7d3c6dcde682435791f638af6aa0eb12/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:53.682206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-abf3bb48-5171-4055-91d1-378acec29e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:53.702624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3bb48-5171-4055-91d1-378acec29e97 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:53.713205 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7b9a9-465d-4844-b112-0248aa5b2edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a572e7d81e2246768d8af3d0f70be94e, user_id=7d3c6dcde682435791f638af6aa0eb12, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:53.719430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7b9a9-465d-4844-b112-0248aa5b2edd admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:53.720702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:53.721614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:53.721923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:53.722353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] PATH_INFO: `/v3/projects/5a8121c7fe714df88e59663263d4f0c2` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:53.723833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8121c7fe714df88e59663263d4f0c2)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:53.726433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:53.753676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1041/2091] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:53 2022] PUT /identity/v3/projects/a572e7d81e2246768d8af3d0f70be94e/users/7d3c6dcde682435791f638af6aa0eb12/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:53.754102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:53.763043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11279a0-20cc-46e2-b600-d4ac07536da5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:53.763475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11279a0-20cc-46e2-b600-d4ac07536da5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:53.765844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11279a0-20cc-46e2-b600-d4ac07536da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:53.842734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8121c7fe714df88e59663263d4f0c2', 'request_id': 'req-aabc087e-3229-4874-b370-15e31d8be467'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:53.843330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c11279a0-20cc-46e2-b600-d4ac07536da5 None None] MFA Rules not processed for user `7d3c6dcde682435791f638af6aa0eb12`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:53.847949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c11279a0-20cc-46e2-b600-d4ac07536da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:53.855157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8121c7fe714df88e59663263d4f0c2', 'request_id': 'req-aabc087e-3229-4874-b370-15e31d8be467'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:53.867444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8121c7fe714df88e59663263d4f0c2', 'request_id': 'req-aabc087e-3229-4874-b370-15e31d8be467'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:53.881772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] The token cache is being invalidate because project 5a8121c7fe714df88e59663263d4f0c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:53:53.883750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:53.885097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-aabc087e-3229-4874-b370-15e31d8be467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:53:53.887668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1051/2092] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:53:53 2022] DELETE /identity/v3/projects/5a8121c7fe714df88e59663263d4f0c2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:53:53.896133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95406a2d-97e7-4c18-9161-2f9ce85802c9 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:53.896965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95406a2d-97e7-4c18-9161-2f9ce85802c9 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:53.897457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95406a2d-97e7-4c18-9161-2f9ce85802c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:53.923956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-95406a2d-97e7-4c18-9161-2f9ce85802c9 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:53.925888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-95406a2d-97e7-4c18-9161-2f9ce85802c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:53.958738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1042/2093] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:53 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:53.963132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11279a0-20cc-46e2-b600-d4ac07536da5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:53.969943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c11279a0-20cc-46e2-b600-d4ac07536da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:54.008041 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:54.013170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1052/2094] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:54.024347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40757d4e-ce71-43c8-97ee-bafe18a2bbda None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.026236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40757d4e-ce71-43c8-97ee-bafe18a2bbda None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.026748 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40757d4e-ce71-43c8-97ee-bafe18a2bbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.082595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-40757d4e-ce71-43c8-97ee-bafe18a2bbda None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:54.083812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-40757d4e-ce71-43c8-97ee-bafe18a2bbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:54.108360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1053/2095] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:54.125906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40757d4e-ce71-43c8-97ee-bafe18a2bbda None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.135813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11279a0-20cc-46e2-b600-d4ac07536da5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.141955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40757d4e-ce71-43c8-97ee-bafe18a2bbda None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.147701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afb29a-feac-4747-a974-6fdc37fe3f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.148791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afb29a-feac-4747-a974-6fdc37fe3f70 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.149303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afb29a-feac-4747-a974-6fdc37fe3f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.149603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afb29a-feac-4747-a974-6fdc37fe3f70 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.150310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afb29a-feac-4747-a974-6fdc37fe3f70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.152557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afb29a-feac-4747-a974-6fdc37fe3f70 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.152864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59abd56-8e08-4b07-bb21-df65328568bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.154578 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59abd56-8e08-4b07-bb21-df65328568bb admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.155114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59abd56-8e08-4b07-bb21-df65328568bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.155597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59abd56-8e08-4b07-bb21-df65328568bb admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.158459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59abd56-8e08-4b07-bb21-df65328568bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:53:54.159037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59abd56-8e08-4b07-bb21-df65328568bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.163217 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59abd56-8e08-4b07-bb21-df65328568bb admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.196814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1054/2096] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:53:54.201667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59abd56-8e08-4b07-bb21-df65328568bb admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.212117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1043/2097] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:53 2022] POST /identity/v3/projects => generated 409 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:54.216266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afb29a-feac-4747-a974-6fdc37fe3f70 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.219611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59abd56-8e08-4b07-bb21-df65328568bb admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.329164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afb29a-feac-4747-a974-6fdc37fe3f70 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.338406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889453f-bf8b-4d8e-ada5-250ed3bf0be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.339685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889453f-bf8b-4d8e-ada5-250ed3bf0be9 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.340675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889453f-bf8b-4d8e-ada5-250ed3bf0be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.341187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889453f-bf8b-4d8e-ada5-250ed3bf0be9 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.342562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889453f-bf8b-4d8e-ada5-250ed3bf0be9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.346334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889453f-bf8b-4d8e-ada5-250ed3bf0be9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.356576 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf20f83-e573-4ab5-b470-6fb831e5b942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.357873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf20f83-e573-4ab5-b470-6fb831e5b942 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.358214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf20f83-e573-4ab5-b470-6fb831e5b942 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.358423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf20f83-e573-4ab5-b470-6fb831e5b942 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.359148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf20f83-e573-4ab5-b470-6fb831e5b942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.364411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf20f83-e573-4ab5-b470-6fb831e5b942 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.378699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1044/2098] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:54 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:53:54.389880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889453f-bf8b-4d8e-ada5-250ed3bf0be9 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.414080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889453f-bf8b-4d8e-ada5-250ed3bf0be9 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.430391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1055/2099] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:54 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:54.430719 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6093477-d5bd-428d-9062-a6725969a31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.431314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf20f83-e573-4ab5-b470-6fb831e5b942 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.431633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6093477-d5bd-428d-9062-a6725969a31f admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.432507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6093477-d5bd-428d-9062-a6725969a31f admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.432507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6093477-d5bd-428d-9062-a6725969a31f admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.433336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6093477-d5bd-428d-9062-a6725969a31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.437522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:54.438589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6093477-d5bd-428d-9062-a6725969a31f admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.464654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1045/2100] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:54.469720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6093477-d5bd-428d-9062-a6725969a31f admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.506361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6093477-d5bd-428d-9062-a6725969a31f admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.536201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf20f83-e573-4ab5-b470-6fb831e5b942 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.547212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f56d4-7156-4f9e-96a1-060fce7f9676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.548506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f56d4-7156-4f9e-96a1-060fce7f9676 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.549124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f56d4-7156-4f9e-96a1-060fce7f9676 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.550003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f56d4-7156-4f9e-96a1-060fce7f9676 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.551198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f56d4-7156-4f9e-96a1-060fce7f9676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.554354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f56d4-7156-4f9e-96a1-060fce7f9676 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.588938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1056/2101] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:54 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:54.593335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f56d4-7156-4f9e-96a1-060fce7f9676 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.615843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f56d4-7156-4f9e-96a1-060fce7f9676 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.628856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38d543-1594-41ed-b88f-855c0dac0ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.630146 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38d543-1594-41ed-b88f-855c0dac0ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.631077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90394c4-0f57-42bd-944d-0c8add22b723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.631988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90394c4-0f57-42bd-944d-0c8add22b723 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.632151 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38d543-1594-41ed-b88f-855c0dac0ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.632409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90394c4-0f57-42bd-944d-0c8add22b723 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.632583 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38d543-1594-41ed-b88f-855c0dac0ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.632732 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90394c4-0f57-42bd-944d-0c8add22b723 admin admin] PATH_INFO: `/v3/projects/0d5e6bdf444f429e82665265e0121807/users/6f6ac01d52dd45ada8de02bd9a0645eb/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.634186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38d543-1594-41ed-b88f-855c0dac0ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.641187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38d543-1594-41ed-b88f-855c0dac0ef2 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.672899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90394c4-0f57-42bd-944d-0c8add22b723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5e6bdf444f429e82665265e0121807, user_id=6f6ac01d52dd45ada8de02bd9a0645eb, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.673367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1057/2102] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:53:54.676285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90394c4-0f57-42bd-944d-0c8add22b723 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.677346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38d543-1594-41ed-b88f-855c0dac0ef2 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.681035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:54.696683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38d543-1594-41ed-b88f-855c0dac0ef2 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.708089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e868510-f618-47d5-9b13-b83092bdcc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.708788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e868510-f618-47d5-9b13-b83092bdcc9e admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.709070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e868510-f618-47d5-9b13-b83092bdcc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.709349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e868510-f618-47d5-9b13-b83092bdcc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.710005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1046/2103] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:54 2022] PUT /identity/v3/projects/0d5e6bdf444f429e82665265e0121807/users/6f6ac01d52dd45ada8de02bd9a0645eb/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:54.711654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90394c4-0f57-42bd-944d-0c8add22b723 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.713146 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e868510-f618-47d5-9b13-b83092bdcc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.721003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e868510-f618-47d5-9b13-b83092bdcc9e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.744789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1058/2104] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:54.750090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e868510-f618-47d5-9b13-b83092bdcc9e admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.784742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e868510-f618-47d5-9b13-b83092bdcc9e admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.849395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90394c4-0f57-42bd-944d-0c8add22b723 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.856076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ab37c-4c72-4fda-8d0c-c18f9e0690af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.856766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ab37c-4c72-4fda-8d0c-c18f9e0690af admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.857045 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ab37c-4c72-4fda-8d0c-c18f9e0690af admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.857373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ab37c-4c72-4fda-8d0c-c18f9e0690af admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.858380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ab37c-4c72-4fda-8d0c-c18f9e0690af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.860371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ab37c-4c72-4fda-8d0c-c18f9e0690af admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.875250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1047/2105] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:54.879885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ab37c-4c72-4fda-8d0c-c18f9e0690af admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:54.905428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ac3c1-895b-4c92-b8ae-35d7f2b1effe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.906936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ac3c1-895b-4c92-b8ae-35d7f2b1effe admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.906936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ac3c1-895b-4c92-b8ae-35d7f2b1effe admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.907113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ac3c1-895b-4c92-b8ae-35d7f2b1effe admin admin] PATH_INFO: `/v3/projects/b37cc2e270be4d79b2cdab03416d830f/users/fd1b54ed85374eb286962c17bf8f7a83/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.907791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ab37c-4c72-4fda-8d0c-c18f9e0690af admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:54.923324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90dfac-31f2-4ec3-a0b8-e472bff84b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:54.926408 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90dfac-31f2-4ec3-a0b8-e472bff84b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.926955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90dfac-31f2-4ec3-a0b8-e472bff84b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.927473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90dfac-31f2-4ec3-a0b8-e472bff84b99 admin admin] PATH_INFO: `/v3/projects/0d5e6bdf444f429e82665265e0121807/users/6f6ac01d52dd45ada8de02bd9a0645eb/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.930684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90dfac-31f2-4ec3-a0b8-e472bff84b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5e6bdf444f429e82665265e0121807, user_id=6f6ac01d52dd45ada8de02bd9a0645eb, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.935352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90dfac-31f2-4ec3-a0b8-e472bff84b99 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.942993 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ac3c1-895b-4c92-b8ae-35d7f2b1effe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37cc2e270be4d79b2cdab03416d830f, user_id=fd1b54ed85374eb286962c17bf8f7a83, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:54.946441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ac3c1-895b-4c92-b8ae-35d7f2b1effe admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:54.950482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1048/2106] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:54 2022] PUT /identity/v3/projects/0d5e6bdf444f429e82665265e0121807/users/6f6ac01d52dd45ada8de02bd9a0645eb/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:54.961834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af439b7-91c1-4178-b209-33b70711f244 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.978060 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1059/2107] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:54 2022] PUT /identity/v3/projects/b37cc2e270be4d79b2cdab03416d830f/users/fd1b54ed85374eb286962c17bf8f7a83/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:54.978383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af439b7-91c1-4178-b209-33b70711f244 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.978383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af439b7-91c1-4178-b209-33b70711f244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:54.988606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54729c38-dc82-400d-b968-e5a7de888503 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:54.989016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54729c38-dc82-400d-b968-e5a7de888503 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:54.990915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54729c38-dc82-400d-b968-e5a7de888503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:55.029621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-4af439b7-91c1-4178-b209-33b70711f244 None None] MFA Rules not processed for user `6f6ac01d52dd45ada8de02bd9a0645eb`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:55.030894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4af439b7-91c1-4178-b209-33b70711f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:55.073298 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-54729c38-dc82-400d-b968-e5a7de888503 None None] MFA Rules not processed for user `fd1b54ed85374eb286962c17bf8f7a83`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:55.074805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-54729c38-dc82-400d-b968-e5a7de888503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:55.188772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:55.240588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1060/2108] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:55.243257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1049/2109] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:54 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:55.256070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54729c38-dc82-400d-b968-e5a7de888503 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:55.259324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-54729c38-dc82-400d-b968-e5a7de888503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:55.260479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af439b7-91c1-4178-b209-33b70711f244 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:55.262371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4af439b7-91c1-4178-b209-33b70711f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:55.375770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af439b7-91c1-4178-b209-33b70711f244 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:55.379717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa70d6-70e5-4178-8154-eae6382d48be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:55.380691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa70d6-70e5-4178-8154-eae6382d48be service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:55.380910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa70d6-70e5-4178-8154-eae6382d48be service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:55.381256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa70d6-70e5-4178-8154-eae6382d48be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:55.395234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa70d6-70e5-4178-8154-eae6382d48be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:55.403397 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa70d6-70e5-4178-8154-eae6382d48be service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:55.422377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54729c38-dc82-400d-b968-e5a7de888503 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:55.437841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878bec3-77db-4045-9c5a-5962a2b1be3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:55.438522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878bec3-77db-4045-9c5a-5962a2b1be3f service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:55.438802 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878bec3-77db-4045-9c5a-5962a2b1be3f service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:55.439081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878bec3-77db-4045-9c5a-5962a2b1be3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:55.455146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1050/2110] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:53:55.461607 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878bec3-77db-4045-9c5a-5962a2b1be3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:55.463209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878bec3-77db-4045-9c5a-5962a2b1be3f service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:55.485667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1061/2111] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:55 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:55.845022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa70d6-70e5-4178-8154-eae6382d48be service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:55.858548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:55.874747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa70d6-70e5-4178-8154-eae6382d48be service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:55.884763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34c018-a57f-4b95-9fd1-11479c9136dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:55.888774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34c018-a57f-4b95-9fd1-11479c9136dc service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:55.889303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34c018-a57f-4b95-9fd1-11479c9136dc service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:55.889815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34c018-a57f-4b95-9fd1-11479c9136dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:56.014694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34c018-a57f-4b95-9fd1-11479c9136dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:56.017424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34c018-a57f-4b95-9fd1-11479c9136dc service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:56.051805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1051/2112] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:53:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:53:56.752386 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5fd5-3caa-41f6-b708-b5f08b233c17 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:56.752900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5fd5-3caa-41f6-b708-b5f08b233c17 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:56.753166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5fd5-3caa-41f6-b708-b5f08b233c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:56.831606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-631a5fd5-3caa-41f6-b708-b5f08b233c17 None None] MFA Rules not processed for user `c815ea64d44a48d89a2f5ff10f3f9ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:53:56.832858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-631a5fd5-3caa-41f6-b708-b5f08b233c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:56.980791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1062/2113] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:53:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:53:56.985454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34c018-a57f-4b95-9fd1-11479c9136dc service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:56.987463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7b34c018-a57f-4b95-9fd1-11479c9136dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:53:57.125031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34c018-a57f-4b95-9fd1-11479c9136dc service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:57.130262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fef804-f790-41fa-a9e6-52bdd19cef29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:57.133758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fef804-f790-41fa-a9e6-52bdd19cef29 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:57.133758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fef804-f790-41fa-a9e6-52bdd19cef29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:57.133758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fef804-f790-41fa-a9e6-52bdd19cef29 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:57.133758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fef804-f790-41fa-a9e6-52bdd19cef29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:57.135369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fef804-f790-41fa-a9e6-52bdd19cef29 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:57.167973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1052/2114] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:53:56 2022] POST /identity/v3/projects => generated 371 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:57.173767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5fd5-3caa-41f6-b708-b5f08b233c17 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:57.190365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5fd5-3caa-41f6-b708-b5f08b233c17 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:57.260731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2297d3-9d07-40d1-9c5c-1ae466a6036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:57.261399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2297d3-9d07-40d1-9c5c-1ae466a6036a admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:57.261716 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2297d3-9d07-40d1-9c5c-1ae466a6036a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:57.261997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2297d3-9d07-40d1-9c5c-1ae466a6036a admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:57.262698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2297d3-9d07-40d1-9c5c-1ae466a6036a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:57.264677 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2297d3-9d07-40d1-9c5c-1ae466a6036a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:57.301509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1063/2115] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:53:57 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:53:57.305936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fef804-f790-41fa-a9e6-52bdd19cef29 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:57.317268 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fef804-f790-41fa-a9e6-52bdd19cef29 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:57.325694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310d8e9-bdaf-48a2-91f8-af7b8668d97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:57.326357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310d8e9-bdaf-48a2-91f8-af7b8668d97b admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:57.326635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310d8e9-bdaf-48a2-91f8-af7b8668d97b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:57.326909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310d8e9-bdaf-48a2-91f8-af7b8668d97b admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:57.327636 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310d8e9-bdaf-48a2-91f8-af7b8668d97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:57.329237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310d8e9-bdaf-48a2-91f8-af7b8668d97b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:57.346492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1053/2116] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:57.351177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2297d3-9d07-40d1-9c5c-1ae466a6036a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:57.362867 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:53:57.367044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2297d3-9d07-40d1-9c5c-1ae466a6036a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:57.379598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711352c0-9eb7-4970-9766-ad3edcb9d258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:57.381091 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711352c0-9eb7-4970-9766-ad3edcb9d258 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:57.381091 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711352c0-9eb7-4970-9766-ad3edcb9d258 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:57.381287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711352c0-9eb7-4970-9766-ad3edcb9d258 admin admin] PATH_INFO: `/v3/projects/43c3f9cd6a0b43eaad0a105608a22c3f/users/eef39c3815b84babb474d2e841f08831/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:57.421789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711352c0-9eb7-4970-9766-ad3edcb9d258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c3f9cd6a0b43eaad0a105608a22c3f, user_id=eef39c3815b84babb474d2e841f08831, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:57.426136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711352c0-9eb7-4970-9766-ad3edcb9d258 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:57.451080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1064/2117] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:57 2022] PUT /identity/v3/projects/43c3f9cd6a0b43eaad0a105608a22c3f/users/eef39c3815b84babb474d2e841f08831/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:53:57.455040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310d8e9-bdaf-48a2-91f8-af7b8668d97b admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:57.472416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310d8e9-bdaf-48a2-91f8-af7b8668d97b admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:57.578686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e08a2-4211-4ddf-aee9-3f92730e5c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:57.579361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e08a2-4211-4ddf-aee9-3f92730e5c30 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:57.579637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e08a2-4211-4ddf-aee9-3f92730e5c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:57.579909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e08a2-4211-4ddf-aee9-3f92730e5c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:57.580630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e08a2-4211-4ddf-aee9-3f92730e5c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:57.582808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e08a2-4211-4ddf-aee9-3f92730e5c30 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:57.600385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1054/2118] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:53:57.603038 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711352c0-9eb7-4970-9766-ad3edcb9d258 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:53:57.615873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711352c0-9eb7-4970-9766-ad3edcb9d258 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:53:57.621412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da965cb-246a-4398-ae33-274d1a2a4741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:53:57.622290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da965cb-246a-4398-ae33-274d1a2a4741 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:53:57.623077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da965cb-246a-4398-ae33-274d1a2a4741 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:53:57.623077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da965cb-246a-4398-ae33-274d1a2a4741 admin admin] PATH_INFO: `/v3/projects/43c3f9cd6a0b43eaad0a105608a22c3f/users/eef39c3815b84babb474d2e841f08831/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:53:57.625601 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da965cb-246a-4398-ae33-274d1a2a4741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c3f9cd6a0b43eaad0a105608a22c3f, user_id=eef39c3815b84babb474d2e841f08831, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:53:57.628493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da965cb-246a-4398-ae33-274d1a2a4741 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:53:57.649197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1065/2119] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:53:57 2022] PUT /identity/v3/projects/43c3f9cd6a0b43eaad0a105608a22c3f/users/eef39c3815b84babb474d2e841f08831/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:02.279704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e08a2-4211-4ddf-aee9-3f92730e5c30 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:02.298067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e08a2-4211-4ddf-aee9-3f92730e5c30 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:02.326279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:02.419789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:02.420875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:02.421321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:02.421785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] PATH_INFO: `/v3/users/fd1b54ed85374eb286962c17bf8f7a83` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:02.423136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd1b54ed85374eb286962c17bf8f7a83)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:02.425314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:02.554922 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd1b54ed85374eb286962c17bf8f7a83'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:02.578850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd1b54ed85374eb286962c17bf8f7a83'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:02.599425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd1b54ed85374eb286962c17bf8f7a83'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:02.612010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd1b54ed85374eb286962c17bf8f7a83'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:02.613666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd1b54ed85374eb286962c17bf8f7a83'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:02.626808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1055/2120] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:02 2022] DELETE /identity/v3/users/fd1b54ed85374eb286962c17bf8f7a83 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:02.632458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ece7e6-e6fe-4ac0-9e9c-96ff6d862533 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:02.633019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ece7e6-e6fe-4ac0-9e9c-96ff6d862533 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:02.633242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ece7e6-e6fe-4ac0-9e9c-96ff6d862533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:02.671056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-59ece7e6-e6fe-4ac0-9e9c-96ff6d862533 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:02.672204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-59ece7e6-e6fe-4ac0-9e9c-96ff6d862533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:02.752746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1066/2121] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:02.777830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:02.779335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:02.949387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5da47-7ce6-48e3-b05b-7c79e80ca80e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:02.954195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21f80e-460e-43b8-a7e0-8d45c26481f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:02.955322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21f80e-460e-43b8-a7e0-8d45c26481f7 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:02.956136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21f80e-460e-43b8-a7e0-8d45c26481f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:02.956642 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe21f80e-460e-43b8-a7e0-8d45c26481f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:02.977149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21f80e-460e-43b8-a7e0-8d45c26481f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:02.980315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe21f80e-460e-43b8-a7e0-8d45c26481f7 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:03.004105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1056/2122] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:03.679859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b0887-8ca8-4623-a52e-72243c0b11c1 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:03.680461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b0887-8ca8-4623-a52e-72243c0b11c1 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:03.680461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b0887-8ca8-4623-a52e-72243c0b11c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:03.726279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21f80e-460e-43b8-a7e0-8d45c26481f7 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:03.728451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fe21f80e-460e-43b8-a7e0-8d45c26481f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:03.745990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe21f80e-460e-43b8-a7e0-8d45c26481f7 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:03.748842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-eb4b0887-8ca8-4623-a52e-72243c0b11c1 None None] MFA Rules not processed for user `eef39c3815b84babb474d2e841f08831`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:03.750165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-eb4b0887-8ca8-4623-a52e-72243c0b11c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:03.756428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:03.758026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:03.758303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:03.758574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] PATH_INFO: `/v3/projects/b37cc2e270be4d79b2cdab03416d830f` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:03.760062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b37cc2e270be4d79b2cdab03416d830f)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:03.762178 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:03.834648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:03.869335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b37cc2e270be4d79b2cdab03416d830f', 'request_id': 'req-a934589f-1548-4274-88bb-f70915abb3c7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:03.881858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b37cc2e270be4d79b2cdab03416d830f', 'request_id': 'req-a934589f-1548-4274-88bb-f70915abb3c7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:03.892373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b37cc2e270be4d79b2cdab03416d830f', 'request_id': 'req-a934589f-1548-4274-88bb-f70915abb3c7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:03.910097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1067/2123] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:03.931011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] The token cache is being invalidate because project b37cc2e270be4d79b2cdab03416d830f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:03.931436 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:03.932172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a934589f-1548-4274-88bb-f70915abb3c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:03.935608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1057/2124] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:03 2022] DELETE /identity/v3/projects/b37cc2e270be4d79b2cdab03416d830f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:03.940846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c07f73-f7fb-4176-9701-5809348915df None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:03.940846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c07f73-f7fb-4176-9701-5809348915df None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:03.941094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c07f73-f7fb-4176-9701-5809348915df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:03.962445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-b6c07f73-f7fb-4176-9701-5809348915df None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:03.963785 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b6c07f73-f7fb-4176-9701-5809348915df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:04.025809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:04.047392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1068/2125] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:04.056183 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12df28-0ad0-45d9-8d39-b880bb703c89 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:04.056501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12df28-0ad0-45d9-8d39-b880bb703c89 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:04.056750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12df28-0ad0-45d9-8d39-b880bb703c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:04.102905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-fe12df28-0ad0-45d9-8d39-b880bb703c89 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:04.104334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fe12df28-0ad0-45d9-8d39-b880bb703c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:04.123513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1058/2126] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:04.124382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c07f73-f7fb-4176-9701-5809348915df None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:04.151533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c07f73-f7fb-4176-9701-5809348915df None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:04.166595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12df28-0ad0-45d9-8d39-b880bb703c89 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:04.169249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fe12df28-0ad0-45d9-8d39-b880bb703c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:04.179766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6274ea-e77a-4b51-9763-acd9efbe4bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:04.182364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6274ea-e77a-4b51-9763-acd9efbe4bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:04.182924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6274ea-e77a-4b51-9763-acd9efbe4bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:04.183535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6274ea-e77a-4b51-9763-acd9efbe4bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:04.185515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6274ea-e77a-4b51-9763-acd9efbe4bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:54:04.187117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6274ea-e77a-4b51-9763-acd9efbe4bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:04.191068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6274ea-e77a-4b51-9763-acd9efbe4bf6 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:04.224931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1069/2127] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:54:04.226070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6274ea-e77a-4b51-9763-acd9efbe4bf6 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:04.249721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6274ea-e77a-4b51-9763-acd9efbe4bf6 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:04.266133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e574fc8-32fb-45b0-b673-a34d19bc0727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:04.267675 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e574fc8-32fb-45b0-b673-a34d19bc0727 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:04.269820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e574fc8-32fb-45b0-b673-a34d19bc0727 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:04.270299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e574fc8-32fb-45b0-b673-a34d19bc0727 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:04.271104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e574fc8-32fb-45b0-b673-a34d19bc0727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:04.274424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e574fc8-32fb-45b0-b673-a34d19bc0727 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:04.317439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1070/2128] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:04 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:04.319009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e574fc8-32fb-45b0-b673-a34d19bc0727 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:04.329974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12df28-0ad0-45d9-8d39-b880bb703c89 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:04.414342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e574fc8-32fb-45b0-b673-a34d19bc0727 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:04.419691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dfadf-612c-404c-85c2-8cff97987188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:04.420559 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dfadf-612c-404c-85c2-8cff97987188 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:04.421002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dfadf-612c-404c-85c2-8cff97987188 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:04.421339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dfadf-612c-404c-85c2-8cff97987188 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:04.422332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dfadf-612c-404c-85c2-8cff97987188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:04.424669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dfadf-612c-404c-85c2-8cff97987188 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:04.441210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6f92f-2908-4de3-80c8-e2822e599892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:04.442349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6f92f-2908-4de3-80c8-e2822e599892 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:04.442469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6f92f-2908-4de3-80c8-e2822e599892 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:04.442796 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6f92f-2908-4de3-80c8-e2822e599892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:04.444895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1ca6f92f-2908-4de3-80c8-e2822e599892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:04.485694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1071/2129] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:04 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:04.487454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dfadf-612c-404c-85c2-8cff97987188 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:04.518801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dfadf-612c-404c-85c2-8cff97987188 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:04.524238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ea91-8b6a-41da-a2fc-5b5d09d53408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:04.525385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ea91-8b6a-41da-a2fc-5b5d09d53408 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:04.525713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ea91-8b6a-41da-a2fc-5b5d09d53408 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:04.526141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ea91-8b6a-41da-a2fc-5b5d09d53408 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:04.527182 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2ea91-8b6a-41da-a2fc-5b5d09d53408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:04.529352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2ea91-8b6a-41da-a2fc-5b5d09d53408 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:04.539087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:04.545637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1072/2130] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:04.557160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ea91-8b6a-41da-a2fc-5b5d09d53408 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:04.560711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6f92f-2908-4de3-80c8-e2822e599892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:04.562911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6f92f-2908-4de3-80c8-e2822e599892 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:04.577338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ea91-8b6a-41da-a2fc-5b5d09d53408 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:04.583423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1059/2131] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:54:04.584064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401e2b8-8b1f-4f02-a63b-2765b6dbb3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:04.585215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401e2b8-8b1f-4f02-a63b-2765b6dbb3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:04.585735 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401e2b8-8b1f-4f02-a63b-2765b6dbb3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:04.586095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401e2b8-8b1f-4f02-a63b-2765b6dbb3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:04.587151 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401e2b8-8b1f-4f02-a63b-2765b6dbb3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:04.589180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401e2b8-8b1f-4f02-a63b-2765b6dbb3f8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:04.607962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1073/2132] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:04.612152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6f92f-2908-4de3-80c8-e2822e599892 service nova] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:04.624872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6f92f-2908-4de3-80c8-e2822e599892 service nova] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:04.630833 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2bf1a-9173-4a6e-ac46-032d9622a952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:04.633401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2bf1a-9173-4a6e-ac46-032d9622a952 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:04.634281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2bf1a-9173-4a6e-ac46-032d9622a952 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:04.634621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2bf1a-9173-4a6e-ac46-032d9622a952 admin admin] PATH_INFO: `/v3/projects/18a31eebe50e4b9fbd5602b74eb3b679/users/03edf287091e45a6a60767f98dfbeb6f/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:04.656432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb2bf1a-9173-4a6e-ac46-032d9622a952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a31eebe50e4b9fbd5602b74eb3b679, user_id=03edf287091e45a6a60767f98dfbeb6f, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:04.659195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb2bf1a-9173-4a6e-ac46-032d9622a952 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:04.683971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1060/2133] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:04 2022] PUT /identity/v3/projects/18a31eebe50e4b9fbd5602b74eb3b679/users/03edf287091e45a6a60767f98dfbeb6f/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:04.694570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83e33c-cf2b-41c0-8110-7ea453c84ea7 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:04.695148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83e33c-cf2b-41c0-8110-7ea453c84ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:04.695346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83e33c-cf2b-41c0-8110-7ea453c84ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:04.733832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-1c83e33c-cf2b-41c0-8110-7ea453c84ea7 None None] MFA Rules not processed for user `03edf287091e45a6a60767f98dfbeb6f`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:04.735866 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-1c83e33c-cf2b-41c0-8110-7ea453c84ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:04.860974 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1074/2134] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:04.883670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2bf1a-9173-4a6e-ac46-032d9622a952 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:04.892361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6eb2bf1a-9173-4a6e-ac46-032d9622a952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:05.090142 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2bf1a-9173-4a6e-ac46-032d9622a952 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:05.103838 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818ab09-3bc7-485f-abcc-0f898e8b1706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:05.121850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818ab09-3bc7-485f-abcc-0f898e8b1706 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:05.122244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818ab09-3bc7-485f-abcc-0f898e8b1706 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:05.122564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818ab09-3bc7-485f-abcc-0f898e8b1706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:05.153147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818ab09-3bc7-485f-abcc-0f898e8b1706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:05.154679 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818ab09-3bc7-485f-abcc-0f898e8b1706 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:05.173073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1061/2135] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:05.224856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83e33c-cf2b-41c0-8110-7ea453c84ea7 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:05.254221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:05.314625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83e33c-cf2b-41c0-8110-7ea453c84ea7 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:05.320781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:05.322470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:05.322470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:05.322686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] PATH_INFO: `/v3/users/03edf287091e45a6a60767f98dfbeb6f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:05.324705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03edf287091e45a6a60767f98dfbeb6f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:05.327145 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:05.420992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03edf287091e45a6a60767f98dfbeb6f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:05.447075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03edf287091e45a6a60767f98dfbeb6f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:05.447808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03edf287091e45a6a60767f98dfbeb6f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:05.481169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03edf287091e45a6a60767f98dfbeb6f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:05.484414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03edf287091e45a6a60767f98dfbeb6f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:05.531911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645bb93-3bf8-4b55-9c5f-a5e90edbced5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:05.532253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645bb93-3bf8-4b55-9c5f-a5e90edbced5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:05.532467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645bb93-3bf8-4b55-9c5f-a5e90edbced5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:05.534847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1075/2136] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:05 2022] DELETE /identity/v3/users/03edf287091e45a6a60767f98dfbeb6f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:05.564676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-8645bb93-3bf8-4b55-9c5f-a5e90edbced5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:05.565763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8645bb93-3bf8-4b55-9c5f-a5e90edbced5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:05.702498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:05.704133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:05.735693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1062/2137] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:05.790477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645bb93-3bf8-4b55-9c5f-a5e90edbced5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:05.791779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8645bb93-3bf8-4b55-9c5f-a5e90edbced5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:05.847816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcafee5-9d9f-412c-8390-269b7e8d3797 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:05.860463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:05.861383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:05.864384 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:05.864776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] PATH_INFO: `/v3/users/7d3c6dcde682435791f638af6aa0eb12` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:05.869836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d3c6dcde682435791f638af6aa0eb12)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:05.871988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:05.938893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645bb93-3bf8-4b55-9c5f-a5e90edbced5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:05.950733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8d9d3-c515-48ba-b9ed-dfc9346e4a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:05.951405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d9d3-c515-48ba-b9ed-dfc9346e4a85 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:05.951689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d9d3-c515-48ba-b9ed-dfc9346e4a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:05.951996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d9d3-c515-48ba-b9ed-dfc9346e4a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:05.957571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:05.972771 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d3c6dcde682435791f638af6aa0eb12'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:05.978894 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8d9d3-c515-48ba-b9ed-dfc9346e4a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:05.980372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8d9d3-c515-48ba-b9ed-dfc9346e4a85 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:05.994841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d3c6dcde682435791f638af6aa0eb12'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:05.995754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d3c6dcde682435791f638af6aa0eb12'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:06.004908 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1063/2138] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:54:06.020892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d3c6dcde682435791f638af6aa0eb12'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:06.021675 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d3c6dcde682435791f638af6aa0eb12'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:06.054092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1076/2139] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:05 2022] DELETE /identity/v3/users/7d3c6dcde682435791f638af6aa0eb12 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:06.059746 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6686bf9-6d8b-4f8c-ba9a-c565cb7bd67b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:06.060054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6686bf9-6d8b-4f8c-ba9a-c565cb7bd67b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:06.060326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6686bf9-6d8b-4f8c-ba9a-c565cb7bd67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:06.098479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e6686bf9-6d8b-4f8c-ba9a-c565cb7bd67b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:06.100923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e6686bf9-6d8b-4f8c-ba9a-c565cb7bd67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:06.113467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:06.223086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1064/2140] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:06.241613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:06.244061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:06.423353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ade0ce-4d55-4510-8898-9dc40c0dc129 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:06.427805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca0399-b229-4849-a70a-050be5b50abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:06.429020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca0399-b229-4849-a70a-050be5b50abf service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:06.429020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca0399-b229-4849-a70a-050be5b50abf service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:06.429020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca0399-b229-4849-a70a-050be5b50abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:06.450816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca0399-b229-4849-a70a-050be5b50abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:06.453569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca0399-b229-4849-a70a-050be5b50abf service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:06.493892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1077/2141] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:07.161040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6686bf9-6d8b-4f8c-ba9a-c565cb7bd67b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:07.162586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e6686bf9-6d8b-4f8c-ba9a-c565cb7bd67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:07.184665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6686bf9-6d8b-4f8c-ba9a-c565cb7bd67b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:07.205923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:07.206884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:07.207161 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:07.208963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] PATH_INFO: `/v3/projects/18a31eebe50e4b9fbd5602b74eb3b679` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:07.212552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a31eebe50e4b9fbd5602b74eb3b679)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:07.217104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:07.353032 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a31eebe50e4b9fbd5602b74eb3b679', 'request_id': 'req-e2ece916-a7ce-403e-86f7-27d1895703a1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:07.369648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a31eebe50e4b9fbd5602b74eb3b679', 'request_id': 'req-e2ece916-a7ce-403e-86f7-27d1895703a1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:07.376532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a31eebe50e4b9fbd5602b74eb3b679', 'request_id': 'req-e2ece916-a7ce-403e-86f7-27d1895703a1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:07.393784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184f9d2-d312-471d-a5d3-55a6f61cd78e None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:07.395961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184f9d2-d312-471d-a5d3-55a6f61cd78e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:07.396521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184f9d2-d312-471d-a5d3-55a6f61cd78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:07.412121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] The token cache is being invalidate because project 18a31eebe50e4b9fbd5602b74eb3b679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:07.412825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:07.415125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:07.418537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1065/2142] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:07 2022] DELETE /identity/v3/projects/18a31eebe50e4b9fbd5602b74eb3b679 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:07.547420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-e184f9d2-d312-471d-a5d3-55a6f61cd78e None None] MFA Rules not processed for user `c815ea64d44a48d89a2f5ff10f3f9ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:07.549126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e184f9d2-d312-471d-a5d3-55a6f61cd78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:07.652575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1078/2143] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:08.043410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:08.045039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:08.170661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ece916-a7ce-403e-86f7-27d1895703a1 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:08.179302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:08.189784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:08.190533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:08.191034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] PATH_INFO: `/v3/projects/a572e7d81e2246768d8af3d0f70be94e` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:08.201619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a572e7d81e2246768d8af3d0f70be94e)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:08.204400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:08.291281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:08.328608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a572e7d81e2246768d8af3d0f70be94e', 'request_id': 'req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:08.349525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a572e7d81e2246768d8af3d0f70be94e', 'request_id': 'req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:08.363862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a572e7d81e2246768d8af3d0f70be94e', 'request_id': 'req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:08.386511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] The token cache is being invalidate because project a572e7d81e2246768d8af3d0f70be94e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:08.387013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:08.387885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a9cc720b-4049-4e4d-b644-a3bf9e4b9bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:08.391362 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1066/2144] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:08 2022] DELETE /identity/v3/projects/a572e7d81e2246768d8af3d0f70be94e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:08.395931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184f9d2-d312-471d-a5d3-55a6f61cd78e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:08.424560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184f9d2-d312-471d-a5d3-55a6f61cd78e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:08.562340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:08.563239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:08.563681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:08.565315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] PATH_INFO: `/v3/users/6f6ac01d52dd45ada8de02bd9a0645eb` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:08.567693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f6ac01d52dd45ada8de02bd9a0645eb)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:08.571651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:08.576234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:08.710338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f6ac01d52dd45ada8de02bd9a0645eb'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:08.733537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f6ac01d52dd45ada8de02bd9a0645eb'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:08.734419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f6ac01d52dd45ada8de02bd9a0645eb'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:08.750529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f6ac01d52dd45ada8de02bd9a0645eb'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:08.751716 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b38c95c-50f4-4d60-9b83-c07209edefd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f6ac01d52dd45ada8de02bd9a0645eb'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:08.769076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1079/2145] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:08 2022] DELETE /identity/v3/users/6f6ac01d52dd45ada8de02bd9a0645eb => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:08.775576 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38902ab4-5eba-4b87-a9f6-ebee390bad7d None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:08.775969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38902ab4-5eba-4b87-a9f6-ebee390bad7d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:08.776319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38902ab4-5eba-4b87-a9f6-ebee390bad7d None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:08.778280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1067/2146] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:54:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:54:08.798621 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2f31e-0143-4a25-ae8a-aeab95d5d2a7 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:08.798999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2f31e-0143-4a25-ae8a-aeab95d5d2a7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:08.799382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2f31e-0143-4a25-ae8a-aeab95d5d2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:08.822981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-2cc2f31e-0143-4a25-ae8a-aeab95d5d2a7 None None] MFA Rules not processed for user `bba7d9205e694f2182d6091061d1219c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:08.824465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2cc2f31e-0143-4a25-ae8a-aeab95d5d2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:08.905452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38902ab4-5eba-4b87-a9f6-ebee390bad7d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:08.908091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-38902ab4-5eba-4b87-a9f6-ebee390bad7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:08.915119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1080/2147] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:54:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:08.932420 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2f31e-0143-4a25-ae8a-aeab95d5d2a7 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:08.935409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2cc2f31e-0143-4a25-ae8a-aeab95d5d2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:08.954133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2f31e-0143-4a25-ae8a-aeab95d5d2a7 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:08.960684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df296f0-659a-48da-b16d-e72568423318 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:08.961295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df296f0-659a-48da-b16d-e72568423318 service placement] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:08.961664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df296f0-659a-48da-b16d-e72568423318 service placement] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:08.961887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df296f0-659a-48da-b16d-e72568423318 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:08.983412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df296f0-659a-48da-b16d-e72568423318 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:08.987604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df296f0-659a-48da-b16d-e72568423318 service placement] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:09.019189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38902ab4-5eba-4b87-a9f6-ebee390bad7d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:09.032079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1081/2148] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:09.053910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:09.055047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.055185 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.061023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] PATH_INFO: `/v3/users/71647f9f93ec4235818d27640058d25c` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.071788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71647f9f93ec4235818d27640058d25c)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:09.078794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:09.197941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71647f9f93ec4235818d27640058d25c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.216953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71647f9f93ec4235818d27640058d25c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.233249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71647f9f93ec4235818d27640058d25c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.262502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71647f9f93ec4235818d27640058d25c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.263344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3ff564db-3d13-49cf-80c4-eaed8eee7990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71647f9f93ec4235818d27640058d25c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.271500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1068/2149] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:08 2022] DELETE /identity/v3/users/71647f9f93ec4235818d27640058d25c => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:09.326958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df296f0-659a-48da-b16d-e72568423318 service placement] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:09.328461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-1df296f0-659a-48da-b16d-e72568423318 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:09.359607 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df296f0-659a-48da-b16d-e72568423318 service placement] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:09.440164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:09.441282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.441778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.442058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] PATH_INFO: `/v3/projects/0d5e6bdf444f429e82665265e0121807` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.443321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5e6bdf444f429e82665265e0121807)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:09.445315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:09.453439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:09.506281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5e6bdf444f429e82665265e0121807', 'request_id': 'req-940bfb26-b6fe-473d-8dea-795a75548722'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.516160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5e6bdf444f429e82665265e0121807', 'request_id': 'req-940bfb26-b6fe-473d-8dea-795a75548722'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.523619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5e6bdf444f429e82665265e0121807', 'request_id': 'req-940bfb26-b6fe-473d-8dea-795a75548722'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.536607 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] The token cache is being invalidate because project 0d5e6bdf444f429e82665265e0121807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:09.537037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.538256 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-940bfb26-b6fe-473d-8dea-795a75548722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.540086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1082/2150] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:09 2022] DELETE /identity/v3/projects/0d5e6bdf444f429e82665265e0121807 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:09.545482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946a243-da4a-4ffb-bd03-4edcfd8eaf56 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.545814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946a243-da4a-4ffb-bd03-4edcfd8eaf56 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.550316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946a243-da4a-4ffb-bd03-4edcfd8eaf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.580777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-e946a243-da4a-4ffb-bd03-4edcfd8eaf56 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:09.582482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e946a243-da4a-4ffb-bd03-4edcfd8eaf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:09.629020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:09.657966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1069/2151] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:54:09.667199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efc7b8-1f63-47c1-8a5c-356c5d9d00d0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.667491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efc7b8-1f63-47c1-8a5c-356c5d9d00d0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.667772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efc7b8-1f63-47c1-8a5c-356c5d9d00d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.689790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c5efc7b8-1f63-47c1-8a5c-356c5d9d00d0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:09.691383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c5efc7b8-1f63-47c1-8a5c-356c5d9d00d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:09.700781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1083/2152] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:54:09.705131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946a243-da4a-4ffb-bd03-4edcfd8eaf56 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:09.707486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efc7b8-1f63-47c1-8a5c-356c5d9d00d0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:09.708847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c5efc7b8-1f63-47c1-8a5c-356c5d9d00d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:09.716278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946a243-da4a-4ffb-bd03-4edcfd8eaf56 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:09.721179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21327a70-6de5-4a37-861b-2cd9b10565ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:09.721893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21327a70-6de5-4a37-861b-2cd9b10565ee admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.722213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21327a70-6de5-4a37-861b-2cd9b10565ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.722452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21327a70-6de5-4a37-861b-2cd9b10565ee admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.723185 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21327a70-6de5-4a37-861b-2cd9b10565ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:54:09.723535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21327a70-6de5-4a37-861b-2cd9b10565ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:09.725214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21327a70-6de5-4a37-861b-2cd9b10565ee admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:09.729809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efc7b8-1f63-47c1-8a5c-356c5d9d00d0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:09.739169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:09.739169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.739169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.739169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] PATH_INFO: `/v3/projects/2347fd8e4a1740fdb98218aa30c150e0` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.739644 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1070/2153] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:54:09.745090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21327a70-6de5-4a37-861b-2cd9b10565ee admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:09.745218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2347fd8e4a1740fdb98218aa30c150e0)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:09.747587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:09.765126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21327a70-6de5-4a37-861b-2cd9b10565ee admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:09.772162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0f28d-b715-4177-82c2-50d829175d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:09.772926 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0f28d-b715-4177-82c2-50d829175d0f admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.773209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0f28d-b715-4177-82c2-50d829175d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.773929 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0f28d-b715-4177-82c2-50d829175d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.774717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0f28d-b715-4177-82c2-50d829175d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:09.778543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0f28d-b715-4177-82c2-50d829175d0f admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:09.810029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2347fd8e4a1740fdb98218aa30c150e0', 'request_id': 'req-204dab52-63c7-46f9-ba54-aa07e001912d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.821003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2347fd8e4a1740fdb98218aa30c150e0', 'request_id': 'req-204dab52-63c7-46f9-ba54-aa07e001912d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.828343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1071/2154] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:09 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:09.832005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2347fd8e4a1740fdb98218aa30c150e0', 'request_id': 'req-204dab52-63c7-46f9-ba54-aa07e001912d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.832349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0f28d-b715-4177-82c2-50d829175d0f admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:09.846888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] The token cache is being invalidate because project 2347fd8e4a1740fdb98218aa30c150e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:09.847253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.847892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-204dab52-63c7-46f9-ba54-aa07e001912d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:09.849991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1084/2155] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:09 2022] DELETE /identity/v3/projects/2347fd8e4a1740fdb98218aa30c150e0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:09.856179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db852a16-3113-48ed-8580-e19033a4b970 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.856464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db852a16-3113-48ed-8580-e19033a4b970 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.856741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db852a16-3113-48ed-8580-e19033a4b970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.891823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-db852a16-3113-48ed-8580-e19033a4b970 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:09.893605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-db852a16-3113-48ed-8580-e19033a4b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:09.921374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0f28d-b715-4177-82c2-50d829175d0f admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:09.926868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b912b-0f4f-4c51-a7ec-1ce7058343a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:09.927621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b912b-0f4f-4c51-a7ec-1ce7058343a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.928037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b912b-0f4f-4c51-a7ec-1ce7058343a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.928256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b912b-0f4f-4c51-a7ec-1ce7058343a3 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.929176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b912b-0f4f-4c51-a7ec-1ce7058343a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:09.931322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b912b-0f4f-4c51-a7ec-1ce7058343a3 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:09.956007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1072/2156] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:09 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:54:09.957904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da572b6e-664d-413c-bce2-3572d1059b6b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.958262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da572b6e-664d-413c-bce2-3572d1059b6b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.958611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da572b6e-664d-413c-bce2-3572d1059b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.972993 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1085/2157] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:09.974590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db852a16-3113-48ed-8580-e19033a4b970 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:09.976181 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-db852a16-3113-48ed-8580-e19033a4b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:09.977525 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-da572b6e-664d-413c-bce2-3572d1059b6b None None] MFA Rules not processed for user `e8bbb1244dca43f8947835d01b69da0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:09.978789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-da572b6e-664d-413c-bce2-3572d1059b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:09.988414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db852a16-3113-48ed-8580-e19033a4b970 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:09.993613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccf78c-5ef7-4cc0-98ae-30d791ee9f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:09.994288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccf78c-5ef7-4cc0-98ae-30d791ee9f5c admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:09.994569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccf78c-5ef7-4cc0-98ae-30d791ee9f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:09.994847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccf78c-5ef7-4cc0-98ae-30d791ee9f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:09.995586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccf78c-5ef7-4cc0-98ae-30d791ee9f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:09.997191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccf78c-5ef7-4cc0-98ae-30d791ee9f5c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.002810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:10.012439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1086/2158] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:10.013918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3422-be2e-4253-a3e5-009073f391dc None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.014295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3422-be2e-4253-a3e5-009073f391dc None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.014591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3422-be2e-4253-a3e5-009073f391dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.035118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6dac3422-be2e-4253-a3e5-009073f391dc None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:10.036322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6dac3422-be2e-4253-a3e5-009073f391dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:10.042599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1073/2159] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:10.043465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da572b6e-664d-413c-bce2-3572d1059b6b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.045155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1087/2160] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:10.048850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac3422-be2e-4253-a3e5-009073f391dc None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.058034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da572b6e-664d-413c-bce2-3572d1059b6b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.058166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac3422-be2e-4253-a3e5-009073f391dc None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.062795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b424e2c-a25b-46e2-892f-aff1cf375a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.063716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b424e2c-a25b-46e2-892f-aff1cf375a42 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.064070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbcf65-5d6b-48af-a29f-0d0b4bc5f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.064211 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b424e2c-a25b-46e2-892f-aff1cf375a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.064292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b424e2c-a25b-46e2-892f-aff1cf375a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.064797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbcf65-5d6b-48af-a29f-0d0b4bc5f0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.065078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b424e2c-a25b-46e2-892f-aff1cf375a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.065194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbcf65-5d6b-48af-a29f-0d0b4bc5f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.065492 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbcf65-5d6b-48af-a29f-0d0b4bc5f0e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.066292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbcf65-5d6b-48af-a29f-0d0b4bc5f0e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:54:10.066603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbcf65-5d6b-48af-a29f-0d0b4bc5f0e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.066790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b424e2c-a25b-46e2-892f-aff1cf375a42 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.070703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbcf65-5d6b-48af-a29f-0d0b4bc5f0e1 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.083997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1074/2161] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:10.084155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1088/2162] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:54:10.086748 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b424e2c-a25b-46e2-892f-aff1cf375a42 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.088341 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbcf65-5d6b-48af-a29f-0d0b4bc5f0e1 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.097432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b424e2c-a25b-46e2-892f-aff1cf375a42 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.099713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbcf65-5d6b-48af-a29f-0d0b4bc5f0e1 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.102524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e797fd-f673-441b-8582-f4fb116db440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.103487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e797fd-f673-441b-8582-f4fb116db440 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.103823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e797fd-f673-441b-8582-f4fb116db440 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.104110 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e797fd-f673-441b-8582-f4fb116db440 admin admin] PATH_INFO: `/v3/projects/560a622c9dc042419eafacb8fce884b6/users/774a875c6cbc4a8bbc1a48a620142a0b/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.106274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee615de5-0aac-4c35-8b75-a52fa990e9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.107004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee615de5-0aac-4c35-8b75-a52fa990e9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.107348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee615de5-0aac-4c35-8b75-a52fa990e9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.107690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee615de5-0aac-4c35-8b75-a52fa990e9b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.108387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee615de5-0aac-4c35-8b75-a52fa990e9b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.110362 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee615de5-0aac-4c35-8b75-a52fa990e9b4 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.144457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1089/2163] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:10 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:54:10.145385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee615de5-0aac-4c35-8b75-a52fa990e9b4 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.151541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e797fd-f673-441b-8582-f4fb116db440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=560a622c9dc042419eafacb8fce884b6, user_id=774a875c6cbc4a8bbc1a48a620142a0b, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.155090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e797fd-f673-441b-8582-f4fb116db440 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.172937 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1075/2164] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:10 2022] PUT /identity/v3/projects/560a622c9dc042419eafacb8fce884b6/users/774a875c6cbc4a8bbc1a48a620142a0b/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:10.183441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60d292-2aef-4b4f-8680-e13921d4aff2 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.183857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60d292-2aef-4b4f-8680-e13921d4aff2 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.184366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60d292-2aef-4b4f-8680-e13921d4aff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.200321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:10.216079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee615de5-0aac-4c35-8b75-a52fa990e9b4 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.219117 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-cc60d292-2aef-4b4f-8680-e13921d4aff2 None None] MFA Rules not processed for user `774a875c6cbc4a8bbc1a48a620142a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:10.220426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-cc60d292-2aef-4b4f-8680-e13921d4aff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:10.317927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a6f4c-5029-4950-84cc-e83794c06c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.318901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a6f4c-5029-4950-84cc-e83794c06c5d admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.319851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a6f4c-5029-4950-84cc-e83794c06c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.320375 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a6f4c-5029-4950-84cc-e83794c06c5d admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.321190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a6f4c-5029-4950-84cc-e83794c06c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.323446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a6f4c-5029-4950-84cc-e83794c06c5d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.328847 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1076/2165] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:54:10.335188 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60d292-2aef-4b4f-8680-e13921d4aff2 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.353297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60d292-2aef-4b4f-8680-e13921d4aff2 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.356811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1090/2166] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:10 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:10.358139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a6f4c-5029-4950-84cc-e83794c06c5d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.362024 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae06ff-11a5-445a-bf82-33c9da4c1687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.362905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae06ff-11a5-445a-bf82-33c9da4c1687 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.363427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae06ff-11a5-445a-bf82-33c9da4c1687 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.363943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae06ff-11a5-445a-bf82-33c9da4c1687 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.365331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae06ff-11a5-445a-bf82-33c9da4c1687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.368963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae06ff-11a5-445a-bf82-33c9da4c1687 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.373089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a6f4c-5029-4950-84cc-e83794c06c5d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.383371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eef643-5bf8-46e5-99c5-f65f1061800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.384076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eef643-5bf8-46e5-99c5-f65f1061800a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.384362 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eef643-5bf8-46e5-99c5-f65f1061800a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.385858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eef643-5bf8-46e5-99c5-f65f1061800a admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.387337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eef643-5bf8-46e5-99c5-f65f1061800a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.389229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eef643-5bf8-46e5-99c5-f65f1061800a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.402484 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1091/2167] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:10.406443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eef643-5bf8-46e5-99c5-f65f1061800a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.406866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1077/2168] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:10 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:10.410281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae06ff-11a5-445a-bf82-33c9da4c1687 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.423374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eef643-5bf8-46e5-99c5-f65f1061800a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.471496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:10.489935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903a01d-4b45-4d3c-ad57-60e57105ee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.491211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a01d-4b45-4d3c-ad57-60e57105ee95 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.491496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a01d-4b45-4d3c-ad57-60e57105ee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.491813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a01d-4b45-4d3c-ad57-60e57105ee95 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.492579 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903a01d-4b45-4d3c-ad57-60e57105ee95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.494462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903a01d-4b45-4d3c-ad57-60e57105ee95 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.502703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae06ff-11a5-445a-bf82-33c9da4c1687 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.505125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1092/2169] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:54:10.509830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903a01d-4b45-4d3c-ad57-60e57105ee95 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.512414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f65ef-c5db-4adc-bdda-04f7d732bda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.513489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f65ef-c5db-4adc-bdda-04f7d732bda6 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.513982 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f65ef-c5db-4adc-bdda-04f7d732bda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.514565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f65ef-c5db-4adc-bdda-04f7d732bda6 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.515440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f65ef-c5db-4adc-bdda-04f7d732bda6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.517457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f65ef-c5db-4adc-bdda-04f7d732bda6 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.525340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903a01d-4b45-4d3c-ad57-60e57105ee95 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.530675 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac6434-2784-4c28-ab94-86215bb357e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.531655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac6434-2784-4c28-ab94-86215bb357e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.531940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac6434-2784-4c28-ab94-86215bb357e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.532215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac6434-2784-4c28-ab94-86215bb357e8 admin admin] PATH_INFO: `/v3/projects/0c4b18ec2c3243c6be2ae8dc34cfc7a0/users/c288a33497c24e31b793130dde455362/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.543427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1078/2170] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:10 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:10.547342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f65ef-c5db-4adc-bdda-04f7d732bda6 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.559274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f65ef-c5db-4adc-bdda-04f7d732bda6 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.562289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac6434-2784-4c28-ab94-86215bb357e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4b18ec2c3243c6be2ae8dc34cfc7a0, user_id=c288a33497c24e31b793130dde455362, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.564282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61685d29-34a7-4962-b121-1681f664acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.565208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac6434-2784-4c28-ab94-86215bb357e8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.565616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61685d29-34a7-4962-b121-1681f664acc2 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.565995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61685d29-34a7-4962-b121-1681f664acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.566390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61685d29-34a7-4962-b121-1681f664acc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.567256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61685d29-34a7-4962-b121-1681f664acc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.569516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61685d29-34a7-4962-b121-1681f664acc2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.581620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1093/2171] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:10 2022] PUT /identity/v3/projects/0c4b18ec2c3243c6be2ae8dc34cfc7a0/users/c288a33497c24e31b793130dde455362/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:10.589730 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1079/2172] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:54:10.594817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac6434-2784-4c28-ab94-86215bb357e8 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.596497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3cdc6-798b-409a-96d5-fe037a576c06 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.596941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3cdc6-798b-409a-96d5-fe037a576c06 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.597552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3cdc6-798b-409a-96d5-fe037a576c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.630633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-aec3cdc6-798b-409a-96d5-fe037a576c06 None None] MFA Rules not processed for user `c288a33497c24e31b793130dde455362`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:10.632546 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-aec3cdc6-798b-409a-96d5-fe037a576c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:10.679464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac6434-2784-4c28-ab94-86215bb357e8 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.684818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eae450-5d22-4dc8-b318-184ccb0a37f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.685861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eae450-5d22-4dc8-b318-184ccb0a37f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.686148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eae450-5d22-4dc8-b318-184ccb0a37f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.686545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eae450-5d22-4dc8-b318-184ccb0a37f5 admin admin] PATH_INFO: `/v3/projects/9732cd60d7b64fe5a0c7c643ee070936/users/9283e33f3c774dcc9a2e7185248673e7/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.700609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:10.708650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eae450-5d22-4dc8-b318-184ccb0a37f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9732cd60d7b64fe5a0c7c643ee070936, user_id=9283e33f3c774dcc9a2e7185248673e7, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.711690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eae450-5d22-4dc8-b318-184ccb0a37f5 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.738543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1094/2173] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:10 2022] PUT /identity/v3/projects/9732cd60d7b64fe5a0c7c643ee070936/users/9283e33f3c774dcc9a2e7185248673e7/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:10.742555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eae450-5d22-4dc8-b318-184ccb0a37f5 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.758934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1080/2174] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:10.763541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3cdc6-798b-409a-96d5-fe037a576c06 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.791408 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3cdc6-798b-409a-96d5-fe037a576c06 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.834130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eae450-5d22-4dc8-b318-184ccb0a37f5 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.841025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa92b8-d969-486f-94dd-44806fcadcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.841965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa92b8-d969-486f-94dd-44806fcadcad admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.842409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa92b8-d969-486f-94dd-44806fcadcad admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.842850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa92b8-d969-486f-94dd-44806fcadcad admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.843767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa92b8-d969-486f-94dd-44806fcadcad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.845559 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa92b8-d969-486f-94dd-44806fcadcad admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.857134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1095/2175] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:54:10.861835 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa92b8-d969-486f-94dd-44806fcadcad admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.872595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca146bca-d62b-40e5-b798-7ad6997a9aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.873379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca146bca-d62b-40e5-b798-7ad6997a9aac admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.873808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca146bca-d62b-40e5-b798-7ad6997a9aac admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.874059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa92b8-d969-486f-94dd-44806fcadcad admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:10.874293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca146bca-d62b-40e5-b798-7ad6997a9aac admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.875098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca146bca-d62b-40e5-b798-7ad6997a9aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.877301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca146bca-d62b-40e5-b798-7ad6997a9aac admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.878824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27687926-fdcd-4df6-b0bb-b6061be5c998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:10.879905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27687926-fdcd-4df6-b0bb-b6061be5c998 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.880354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27687926-fdcd-4df6-b0bb-b6061be5c998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.880577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27687926-fdcd-4df6-b0bb-b6061be5c998 admin admin] PATH_INFO: `/v3/projects/9732cd60d7b64fe5a0c7c643ee070936/users/9283e33f3c774dcc9a2e7185248673e7/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.885510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27687926-fdcd-4df6-b0bb-b6061be5c998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9732cd60d7b64fe5a0c7c643ee070936, user_id=9283e33f3c774dcc9a2e7185248673e7, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:10.886519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27687926-fdcd-4df6-b0bb-b6061be5c998 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:10.906296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1096/2176] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:10 2022] PUT /identity/v3/projects/9732cd60d7b64fe5a0c7c643ee070936/users/9283e33f3c774dcc9a2e7185248673e7/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:10.909392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1081/2177] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:10 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:10.910480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27687926-fdcd-4df6-b0bb-b6061be5c998 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:10.917602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fa0ec-7f94-4874-8ed9-1f59807bf83f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:10.918107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fa0ec-7f94-4874-8ed9-1f59807bf83f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:10.918686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fa0ec-7f94-4874-8ed9-1f59807bf83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:10.940699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:10.964122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-749fa0ec-7f94-4874-8ed9-1f59807bf83f None None] MFA Rules not processed for user `9283e33f3c774dcc9a2e7185248673e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:10.965491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-749fa0ec-7f94-4874-8ed9-1f59807bf83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:10.996431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27687926-fdcd-4df6-b0bb-b6061be5c998 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:11.102135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c160d-ad5c-495f-81f2-09c9bfb71993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:11.102945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c160d-ad5c-495f-81f2-09c9bfb71993 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:11.103331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c160d-ad5c-495f-81f2-09c9bfb71993 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:11.103746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c160d-ad5c-495f-81f2-09c9bfb71993 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:11.104550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c160d-ad5c-495f-81f2-09c9bfb71993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:11.106774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c160d-ad5c-495f-81f2-09c9bfb71993 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:11.126699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1082/2178] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:11.145068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fa0ec-7f94-4874-8ed9-1f59807bf83f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:11.145611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1097/2179] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:10 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:11.147988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c160d-ad5c-495f-81f2-09c9bfb71993 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:11.151008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-028c160d-ad5c-495f-81f2-09c9bfb71993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:11.161789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fa0ec-7f94-4874-8ed9-1f59807bf83f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:11.177992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325637be-3cf2-4c4b-afa1-016f08784f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:11.177992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325637be-3cf2-4c4b-afa1-016f08784f51 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:11.177992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325637be-3cf2-4c4b-afa1-016f08784f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:11.177992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325637be-3cf2-4c4b-afa1-016f08784f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:11.180023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325637be-3cf2-4c4b-afa1-016f08784f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:11.180023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325637be-3cf2-4c4b-afa1-016f08784f51 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:11.186219 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1083/2180] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:11.190820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325637be-3cf2-4c4b-afa1-016f08784f51 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:11.203417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325637be-3cf2-4c4b-afa1-016f08784f51 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:11.209121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6d926-9e66-4c7c-9d5a-fded1570c5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:11.210375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6d926-9e66-4c7c-9d5a-fded1570c5d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:11.210718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6d926-9e66-4c7c-9d5a-fded1570c5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:11.211006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6d926-9e66-4c7c-9d5a-fded1570c5d0 admin admin] PATH_INFO: `/v3/projects/b04c6f68c110410fae09e1971fa9fd65/users/fcc03a605fd744a7b091b1c00904fbae/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:11.228161 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6d926-9e66-4c7c-9d5a-fded1570c5d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04c6f68c110410fae09e1971fa9fd65, user_id=fcc03a605fd744a7b091b1c00904fbae, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:11.230932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6d926-9e66-4c7c-9d5a-fded1570c5d0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:11.232025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c160d-ad5c-495f-81f2-09c9bfb71993 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:11.236740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b6229-8699-4289-9a22-17324723346f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:11.237604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b6229-8699-4289-9a22-17324723346f service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:11.238067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b6229-8699-4289-9a22-17324723346f service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:11.238347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b6229-8699-4289-9a22-17324723346f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:11.245391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1084/2181] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:11 2022] PUT /identity/v3/projects/b04c6f68c110410fae09e1971fa9fd65/users/fcc03a605fd744a7b091b1c00904fbae/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:11.249181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6d926-9e66-4c7c-9d5a-fded1570c5d0 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:11.249678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b6229-8699-4289-9a22-17324723346f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:11.251474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b6229-8699-4289-9a22-17324723346f service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:11.264656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1098/2182] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:11.279798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:11.356189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6d926-9e66-4c7c-9d5a-fded1570c5d0 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:11.362857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fa1b7-124f-4cae-ae67-3cbe11fe54e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:11.363531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fa1b7-124f-4cae-ae67-3cbe11fe54e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:11.363816 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fa1b7-124f-4cae-ae67-3cbe11fe54e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:11.364177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fa1b7-124f-4cae-ae67-3cbe11fe54e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:11.364916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fa1b7-124f-4cae-ae67-3cbe11fe54e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:11.368039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fa1b7-124f-4cae-ae67-3cbe11fe54e1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:11.384146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1085/2183] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:11.388809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b6229-8699-4289-9a22-17324723346f service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:11.412942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b6229-8699-4289-9a22-17324723346f service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:11.419555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69894a2f-716b-4230-8b9a-8d5898d63947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:11.420517 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69894a2f-716b-4230-8b9a-8d5898d63947 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:11.420798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69894a2f-716b-4230-8b9a-8d5898d63947 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:11.421074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69894a2f-716b-4230-8b9a-8d5898d63947 admin admin] PATH_INFO: `/v3/projects/b04c6f68c110410fae09e1971fa9fd65/users/fcc03a605fd744a7b091b1c00904fbae/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:11.423101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69894a2f-716b-4230-8b9a-8d5898d63947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04c6f68c110410fae09e1971fa9fd65, user_id=fcc03a605fd744a7b091b1c00904fbae, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:11.426227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69894a2f-716b-4230-8b9a-8d5898d63947 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:11.439684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1099/2184] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:11 2022] PUT /identity/v3/projects/b04c6f68c110410fae09e1971fa9fd65/users/fcc03a605fd744a7b091b1c00904fbae/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:11.450395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abecb4-9586-4cbb-b53f-250a5b5b1ca3 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:11.450801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abecb4-9586-4cbb-b53f-250a5b5b1ca3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:11.451965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abecb4-9586-4cbb-b53f-250a5b5b1ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:11.486949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-66abecb4-9586-4cbb-b53f-250a5b5b1ca3 None None] MFA Rules not processed for user `fcc03a605fd744a7b091b1c00904fbae`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:11.488715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-66abecb4-9586-4cbb-b53f-250a5b5b1ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:11.577055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1086/2185] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:11.591653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69894a2f-716b-4230-8b9a-8d5898d63947 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:11.593532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-69894a2f-716b-4230-8b9a-8d5898d63947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:11.651417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:11.656040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69894a2f-716b-4230-8b9a-8d5898d63947 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:11.665109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01091f30-f168-4d41-87e4-4b7cf06ad93e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:11.666340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01091f30-f168-4d41-87e4-4b7cf06ad93e service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:11.666938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01091f30-f168-4d41-87e4-4b7cf06ad93e service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:11.667377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01091f30-f168-4d41-87e4-4b7cf06ad93e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:11.682211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01091f30-f168-4d41-87e4-4b7cf06ad93e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:11.683648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01091f30-f168-4d41-87e4-4b7cf06ad93e service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:11.693368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1100/2186] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:11.969174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abecb4-9586-4cbb-b53f-250a5b5b1ca3 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:12.035767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abecb4-9586-4cbb-b53f-250a5b5b1ca3 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:12.040899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:12.041897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:12.042297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:12.042576 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] PATH_INFO: `/v3/users/774a875c6cbc4a8bbc1a48a620142a0b` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:12.043784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=774a875c6cbc4a8bbc1a48a620142a0b)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:12.045820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:12.099967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:12.106876 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '774a875c6cbc4a8bbc1a48a620142a0b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:12.124267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '774a875c6cbc4a8bbc1a48a620142a0b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:12.133317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '774a875c6cbc4a8bbc1a48a620142a0b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:12.144497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '774a875c6cbc4a8bbc1a48a620142a0b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:12.145280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '774a875c6cbc4a8bbc1a48a620142a0b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:12.154228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1087/2187] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:11 2022] DELETE /identity/v3/users/774a875c6cbc4a8bbc1a48a620142a0b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:12.157986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:12.158597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1edd87-ae92-4773-b40a-d1fc8704a221 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:12.158883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1edd87-ae92-4773-b40a-d1fc8704a221 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:12.159155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1edd87-ae92-4773-b40a-d1fc8704a221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:12.159395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:12.176655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6a1edd87-ae92-4773-b40a-d1fc8704a221 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:12.177865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6a1edd87-ae92-4773-b40a-d1fc8704a221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:12.239785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebd995-a4d1-48da-b8b7-4c2ce112bfe1 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:12.245372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:12.246390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:12.246716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:12.247055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] PATH_INFO: `/v3/users/c288a33497c24e31b793130dde455362` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:12.247708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1101/2188] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:12.248604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c288a33497c24e31b793130dde455362)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:12.250790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:12.260431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1edd87-ae92-4773-b40a-d1fc8704a221 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:12.262152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6a1edd87-ae92-4773-b40a-d1fc8704a221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:12.314040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c288a33497c24e31b793130dde455362'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:12.327652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c288a33497c24e31b793130dde455362'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:12.343189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c288a33497c24e31b793130dde455362'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:12.354642 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c288a33497c24e31b793130dde455362'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:12.355508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c99e26d6-d9c7-416b-9b5f-68abc377a30a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c288a33497c24e31b793130dde455362'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:12.356000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1edd87-ae92-4773-b40a-d1fc8704a221 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:12.362206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e80d4-7e1d-4db1-bb17-ca1f7e2f80b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:12.362802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1088/2189] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:12 2022] DELETE /identity/v3/users/c288a33497c24e31b793130dde455362 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:12.363555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e80d4-7e1d-4db1-bb17-ca1f7e2f80b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:12.364011 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e80d4-7e1d-4db1-bb17-ca1f7e2f80b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:12.364471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e80d4-7e1d-4db1-bb17-ca1f7e2f80b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:12.367390 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-799e80d4-7e1d-4db1-bb17-ca1f7e2f80b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:12.368222 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e46109-cc4b-4014-b45a-1be75601564e None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:12.368608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e46109-cc4b-4014-b45a-1be75601564e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:12.369020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e46109-cc4b-4014-b45a-1be75601564e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:12.383822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e80d4-7e1d-4db1-bb17-ca1f7e2f80b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:12.385360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e80d4-7e1d-4db1-bb17-ca1f7e2f80b2 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:12.393683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-46e46109-cc4b-4014-b45a-1be75601564e None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:12.395538 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-46e46109-cc4b-4014-b45a-1be75601564e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:12.398770 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1102/2190] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:54:12.437150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:12.476103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1089/2191] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:12.487582 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e80d4-7e1d-4db1-bb17-ca1f7e2f80b2 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:12.489452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-799e80d4-7e1d-4db1-bb17-ca1f7e2f80b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:12.506000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e80d4-7e1d-4db1-bb17-ca1f7e2f80b2 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:12.565769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb105d6-2931-4c71-814f-43dd0a67676c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:12.566349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb105d6-2931-4c71-814f-43dd0a67676c service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:12.566630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb105d6-2931-4c71-814f-43dd0a67676c service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:12.566904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb105d6-2931-4c71-814f-43dd0a67676c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:12.577401 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb105d6-2931-4c71-814f-43dd0a67676c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:12.578878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb105d6-2931-4c71-814f-43dd0a67676c service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:12.594511 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1103/2192] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:12.864154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e46109-cc4b-4014-b45a-1be75601564e None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:12.866318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-46e46109-cc4b-4014-b45a-1be75601564e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:12.898306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e46109-cc4b-4014-b45a-1be75601564e None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:12.905348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:12.906523 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:12.906815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:12.907092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] PATH_INFO: `/v3/projects/560a622c9dc042419eafacb8fce884b6` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:12.908737 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=560a622c9dc042419eafacb8fce884b6)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:12.910832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:13.018974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '560a622c9dc042419eafacb8fce884b6', 'request_id': 'req-f1bd89f9-34cf-4992-bc09-0735c1448d51'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.029562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '560a622c9dc042419eafacb8fce884b6', 'request_id': 'req-f1bd89f9-34cf-4992-bc09-0735c1448d51'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.035941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '560a622c9dc042419eafacb8fce884b6', 'request_id': 'req-f1bd89f9-34cf-4992-bc09-0735c1448d51'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.048629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] The token cache is being invalidate because project 560a622c9dc042419eafacb8fce884b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:13.048920 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.049600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.051187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1090/2193] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:12 2022] DELETE /identity/v3/projects/560a622c9dc042419eafacb8fce884b6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:13.055348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb105d6-2931-4c71-814f-43dd0a67676c service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:13.065933 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb105d6-2931-4c71-814f-43dd0a67676c service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:13.111410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:13.137136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:13.138991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:13.145633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:13.148146 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:13.148886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:13.149266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] PATH_INFO: `/v3/users/9283e33f3c774dcc9a2e7185248673e7` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:13.152055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9283e33f3c774dcc9a2e7185248673e7)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:13.159717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:13.159929 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd89f9-34cf-4992-bc09-0735c1448d51 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:13.161665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:13.162706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:13.163198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:13.163559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] PATH_INFO: `/v3/projects/0c4b18ec2c3243c6be2ae8dc34cfc7a0` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:13.165026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4b18ec2c3243c6be2ae8dc34cfc7a0)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:13.167137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:13.210811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9283e33f3c774dcc9a2e7185248673e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.218334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4b18ec2c3243c6be2ae8dc34cfc7a0', 'request_id': 'req-d82c01d6-e7fc-4d25-a347-868435bea638'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.220775 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9283e33f3c774dcc9a2e7185248673e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.221387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9283e33f3c774dcc9a2e7185248673e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.231780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4b18ec2c3243c6be2ae8dc34cfc7a0', 'request_id': 'req-d82c01d6-e7fc-4d25-a347-868435bea638'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.244919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9283e33f3c774dcc9a2e7185248673e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.245715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9283e33f3c774dcc9a2e7185248673e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.250758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4b18ec2c3243c6be2ae8dc34cfc7a0', 'request_id': 'req-d82c01d6-e7fc-4d25-a347-868435bea638'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.268314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1104/2194] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:13 2022] DELETE /identity/v3/users/9283e33f3c774dcc9a2e7185248673e7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:13.269543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] The token cache is being invalidate because project 0c4b18ec2c3243c6be2ae8dc34cfc7a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:13.269543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.269543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.270609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1091/2195] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:13 2022] DELETE /identity/v3/projects/0c4b18ec2c3243c6be2ae8dc34cfc7a0 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:13.275093 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:13.276480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:13.346669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940fc4e-1f66-4fd2-ad22-ab9b461d33f2 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:13.351567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:13.352454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:13.352735 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:13.353009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] PATH_INFO: `/v3/users/fcc03a605fd744a7b091b1c00904fbae` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:13.354273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc03a605fd744a7b091b1c00904fbae)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:13.356181 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:13.422676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc03a605fd744a7b091b1c00904fbae'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.431848 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc03a605fd744a7b091b1c00904fbae'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.432500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc03a605fd744a7b091b1c00904fbae'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.438258 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:13.454527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc03a605fd744a7b091b1c00904fbae'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.456080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc03a605fd744a7b091b1c00904fbae'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:13.479261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1105/2196] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:13 2022] DELETE /identity/v3/users/fcc03a605fd744a7b091b1c00904fbae => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:13.823275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:13.824861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:13.894306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:13.915447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c01d6-e7fc-4d25-a347-868435bea638 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:13.982329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:13.983224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:13.983503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:13.984258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] PATH_INFO: `/v3/projects/9732cd60d7b64fe5a0c7c643ee070936` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:13.985382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9732cd60d7b64fe5a0c7c643ee070936)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:13.987463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.043574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.045702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:14.064175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf4ef0-cb71-4e50-b00d-f2242b59c85f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.070883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9732cd60d7b64fe5a0c7c643ee070936', 'request_id': 'req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:14.089640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9732cd60d7b64fe5a0c7c643ee070936', 'request_id': 'req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:14.101064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9732cd60d7b64fe5a0c7c643ee070936', 'request_id': 'req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:14.121453 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] The token cache is being invalidate because project 9732cd60d7b64fe5a0c7c643ee070936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:14.121783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:14.122457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f5c28e33-097e-4e2d-a4d2-a2fac1addc00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:14.124253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1092/2197] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:13 2022] DELETE /identity/v3/projects/9732cd60d7b64fe5a0c7c643ee070936 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:14.128967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab7219-0291-4179-a810-66b4e1fdc35f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.129191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab7219-0291-4179-a810-66b4e1fdc35f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.129474 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab7219-0291-4179-a810-66b4e1fdc35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.140865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.141852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.143456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.143860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] PATH_INFO: `/v3/projects/b04c6f68c110410fae09e1971fa9fd65` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.145689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04c6f68c110410fae09e1971fa9fd65)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.148293 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.151319 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-21ab7219-0291-4179-a810-66b4e1fdc35f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:14.152881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-21ab7219-0291-4179-a810-66b4e1fdc35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:14.165620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1093/2198] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:54:14.171087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023a3fa-a802-4f13-a691-bb2a440817b1 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.171571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023a3fa-a802-4f13-a691-bb2a440817b1 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.171903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023a3fa-a802-4f13-a691-bb2a440817b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.193732 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-0023a3fa-a802-4f13-a691-bb2a440817b1 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:14.195464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0023a3fa-a802-4f13-a691-bb2a440817b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:14.210682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04c6f68c110410fae09e1971fa9fd65', 'request_id': 'req-4b6a0226-139a-4e20-809a-cff55da5585f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:14.219077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04c6f68c110410fae09e1971fa9fd65', 'request_id': 'req-4b6a0226-139a-4e20-809a-cff55da5585f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:14.226033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04c6f68c110410fae09e1971fa9fd65', 'request_id': 'req-4b6a0226-139a-4e20-809a-cff55da5585f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:14.242402 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] The token cache is being invalidate because project b04c6f68c110410fae09e1971fa9fd65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:14.242694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:14.243369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-4b6a0226-139a-4e20-809a-cff55da5585f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:14.248298 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1106/2199] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:14 2022] DELETE /identity/v3/projects/b04c6f68c110410fae09e1971fa9fd65 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:14.250180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787be9-ba4a-47b2-ac6b-eec2d72a1c8e None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.250534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787be9-ba4a-47b2-ac6b-eec2d72a1c8e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.250872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787be9-ba4a-47b2-ac6b-eec2d72a1c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.272043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f1787be9-ba4a-47b2-ac6b-eec2d72a1c8e None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:14.273346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f1787be9-ba4a-47b2-ac6b-eec2d72a1c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:14.275581 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1094/2200] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:14.280367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023a3fa-a802-4f13-a691-bb2a440817b1 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.281966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1107/2201] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:14.291250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023a3fa-a802-4f13-a691-bb2a440817b1 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.291410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469a88e-76e2-4429-8bd3-87529a24c8b2 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.291695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469a88e-76e2-4429-8bd3-87529a24c8b2 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.291875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469a88e-76e2-4429-8bd3-87529a24c8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.296461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbd32c-6759-495a-b8ab-976a99fd9ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.297133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd32c-6759-495a-b8ab-976a99fd9ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.297418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd32c-6759-495a-b8ab-976a99fd9ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.297738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbd32c-6759-495a-b8ab-976a99fd9ae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.298480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbd32c-6759-495a-b8ab-976a99fd9ae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:54:14.298787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbd32c-6759-495a-b8ab-976a99fd9ae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.300510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbd32c-6759-495a-b8ab-976a99fd9ae5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.314199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1095/2202] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:54:14.315570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-0469a88e-76e2-4429-8bd3-87529a24c8b2 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:14.316903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0469a88e-76e2-4429-8bd3-87529a24c8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:14.323196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbd32c-6759-495a-b8ab-976a99fd9ae5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.328516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:14.329924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1108/2203] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:54:14.334047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469a88e-76e2-4429-8bd3-87529a24c8b2 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.337914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbd32c-6759-495a-b8ab-976a99fd9ae5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.343622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dba02d-9bb7-41a7-bfe6-645f1524c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.343921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469a88e-76e2-4429-8bd3-87529a24c8b2 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.344277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dba02d-9bb7-41a7-bfe6-645f1524c4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.344561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dba02d-9bb7-41a7-bfe6-645f1524c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.344943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dba02d-9bb7-41a7-bfe6-645f1524c4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.345556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dba02d-9bb7-41a7-bfe6-645f1524c4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.347485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dba02d-9bb7-41a7-bfe6-645f1524c4b5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.348506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b0a1c-af3c-4c7c-a79a-5f3abac1e366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.349275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b0a1c-af3c-4c7c-a79a-5f3abac1e366 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.349558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b0a1c-af3c-4c7c-a79a-5f3abac1e366 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.350467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b0a1c-af3c-4c7c-a79a-5f3abac1e366 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.351768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b0a1c-af3c-4c7c-a79a-5f3abac1e366 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:54:14.352056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b0a1c-af3c-4c7c-a79a-5f3abac1e366 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.353687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b0a1c-af3c-4c7c-a79a-5f3abac1e366 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.366628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1109/2204] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:54:14.371269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b0a1c-af3c-4c7c-a79a-5f3abac1e366 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.379193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1096/2205] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:14 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:54:14.380648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dba02d-9bb7-41a7-bfe6-645f1524c4b5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.381541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b0a1c-af3c-4c7c-a79a-5f3abac1e366 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.420787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:14.442846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9710830-4de0-45fc-8f68-4529d83badb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.443508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9710830-4de0-45fc-8f68-4529d83badb5 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.443814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9710830-4de0-45fc-8f68-4529d83badb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.444078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9710830-4de0-45fc-8f68-4529d83badb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.444868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9710830-4de0-45fc-8f68-4529d83badb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.446998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9710830-4de0-45fc-8f68-4529d83badb5 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.449318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dba02d-9bb7-41a7-bfe6-645f1524c4b5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.454067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a56ca-6f63-4688-886e-1939d78ced1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.454731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a56ca-6f63-4688-886e-1939d78ced1f admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.455020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a56ca-6f63-4688-886e-1939d78ced1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.455284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a56ca-6f63-4688-886e-1939d78ced1f admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.455965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a56ca-6f63-4688-886e-1939d78ced1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.457909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a56ca-6f63-4688-886e-1939d78ced1f admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.472823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1110/2206] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:14 2022] POST /identity/v3/projects => generated 357 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:54:14.477687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9710830-4de0-45fc-8f68-4529d83badb5 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.479766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1097/2207] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:14 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:14.483914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a56ca-6f63-4688-886e-1939d78ced1f admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.497027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a56ca-6f63-4688-886e-1939d78ced1f admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.558157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9710830-4de0-45fc-8f68-4529d83badb5 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.563274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad7335-6b40-4be0-a5e9-b16e5402e74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.564041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad7335-6b40-4be0-a5e9-b16e5402e74d admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.564387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad7335-6b40-4be0-a5e9-b16e5402e74d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.564764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad7335-6b40-4be0-a5e9-b16e5402e74d admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.565513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad7335-6b40-4be0-a5e9-b16e5402e74d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.567522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad7335-6b40-4be0-a5e9-b16e5402e74d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.568469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6df01f-039d-45cd-847d-657990599381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.569875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6df01f-039d-45cd-847d-657990599381 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.569995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6df01f-039d-45cd-847d-657990599381 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.570263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6df01f-039d-45cd-847d-657990599381 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.570994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6df01f-039d-45cd-847d-657990599381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.572741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6df01f-039d-45cd-847d-657990599381 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.586395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1098/2208] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:54:14.592133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6df01f-039d-45cd-847d-657990599381 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.597385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1111/2209] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:14 2022] POST /identity/v3/users => generated 279 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:14.602081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad7335-6b40-4be0-a5e9-b16e5402e74d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.603809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6df01f-039d-45cd-847d-657990599381 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.608669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb2241-db71-4d05-843f-78bd0958ef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.609623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb2241-db71-4d05-843f-78bd0958ef5a admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.609623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb2241-db71-4d05-843f-78bd0958ef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.609947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb2241-db71-4d05-843f-78bd0958ef5a admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.610595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb2241-db71-4d05-843f-78bd0958ef5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.612191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb2241-db71-4d05-843f-78bd0958ef5a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.613845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad7335-6b40-4be0-a5e9-b16e5402e74d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.619464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f173ce-8a8a-412c-b616-ef6d2694135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.620124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f173ce-8a8a-412c-b616-ef6d2694135a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.620405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f173ce-8a8a-412c-b616-ef6d2694135a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.621280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f173ce-8a8a-412c-b616-ef6d2694135a admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.622052 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f173ce-8a8a-412c-b616-ef6d2694135a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.623723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f173ce-8a8a-412c-b616-ef6d2694135a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.626805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1099/2210] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:54:14.632254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb2241-db71-4d05-843f-78bd0958ef5a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.641892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1112/2211] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:14.646155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f173ce-8a8a-412c-b616-ef6d2694135a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.649638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb2241-db71-4d05-843f-78bd0958ef5a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.659286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b80d3f-5eac-4bcc-bddc-690bf12afad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.659550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f173ce-8a8a-412c-b616-ef6d2694135a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.660189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b80d3f-5eac-4bcc-bddc-690bf12afad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.660551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b80d3f-5eac-4bcc-bddc-690bf12afad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.660788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b80d3f-5eac-4bcc-bddc-690bf12afad6 admin admin] PATH_INFO: `/v3/projects/3635a147c8214c38b96de99037249a13/users/c048704fabf546bfa676bdc9524f3e75/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.665046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5fa7a-b09e-4872-ab35-c212401c076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.665819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5fa7a-b09e-4872-ab35-c212401c076d admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.666180 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5fa7a-b09e-4872-ab35-c212401c076d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.666500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5fa7a-b09e-4872-ab35-c212401c076d admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.667288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5fa7a-b09e-4872-ab35-c212401c076d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.668961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5fa7a-b09e-4872-ab35-c212401c076d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.680367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b80d3f-5eac-4bcc-bddc-690bf12afad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3635a147c8214c38b96de99037249a13, user_id=c048704fabf546bfa676bdc9524f3e75, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.683164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b80d3f-5eac-4bcc-bddc-690bf12afad6 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.685442 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1113/2212] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:54:14.689418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5fa7a-b09e-4872-ab35-c212401c076d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.703578 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1100/2213] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:14 2022] PUT /identity/v3/projects/3635a147c8214c38b96de99037249a13/users/c048704fabf546bfa676bdc9524f3e75/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:14.710883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5fa7a-b09e-4872-ab35-c212401c076d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:14.714567 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae35d7e-8bf5-4d47-bdaf-ee917e44a3e6 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.715099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae35d7e-8bf5-4d47-bdaf-ee917e44a3e6 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.715495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae35d7e-8bf5-4d47-bdaf-ee917e44a3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.766404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-5ae35d7e-8bf5-4d47-bdaf-ee917e44a3e6 None None] MFA Rules not processed for user `c048704fabf546bfa676bdc9524f3e75`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:14.768085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5ae35d7e-8bf5-4d47-bdaf-ee917e44a3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:14.808361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:14.830590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e770000-27e1-4c28-8103-e09ffeae3725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:14.831505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e770000-27e1-4c28-8103-e09ffeae3725 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.831787 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e770000-27e1-4c28-8103-e09ffeae3725 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.832132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e770000-27e1-4c28-8103-e09ffeae3725 admin admin] PATH_INFO: `/v3/projects/5e80831f3b804a9b8eecd37cb006f195/users/69e73e9ed6b548aa811cb01243be18e1/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.855575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e770000-27e1-4c28-8103-e09ffeae3725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e80831f3b804a9b8eecd37cb006f195, user_id=69e73e9ed6b548aa811cb01243be18e1, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:14.869018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e770000-27e1-4c28-8103-e09ffeae3725 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:14.892392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1114/2214] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:14 2022] PUT /identity/v3/projects/5e80831f3b804a9b8eecd37cb006f195/users/69e73e9ed6b548aa811cb01243be18e1/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:14.902726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb8300-ddbd-4ca2-99ff-3cde47952fcd None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:14.903925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb8300-ddbd-4ca2-99ff-3cde47952fcd None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:14.904209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb8300-ddbd-4ca2-99ff-3cde47952fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:14.907536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1101/2215] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:14.923905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae35d7e-8bf5-4d47-bdaf-ee917e44a3e6 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:14.926901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5ae35d7e-8bf5-4d47-bdaf-ee917e44a3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:14.952011 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-34cb8300-ddbd-4ca2-99ff-3cde47952fcd None None] MFA Rules not processed for user `69e73e9ed6b548aa811cb01243be18e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:14.953606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-34cb8300-ddbd-4ca2-99ff-3cde47952fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:14.959509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:15.000760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae35d7e-8bf5-4d47-bdaf-ee917e44a3e6 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:15.041472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1115/2216] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:14 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:54:15.046348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb8300-ddbd-4ca2-99ff-3cde47952fcd None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:15.058206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dd7af-ef2f-4118-89c7-838a4847f849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:15.059179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dd7af-ef2f-4118-89c7-838a4847f849 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:15.059654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dd7af-ef2f-4118-89c7-838a4847f849 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:15.060137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dd7af-ef2f-4118-89c7-838a4847f849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:15.121608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb8300-ddbd-4ca2-99ff-3cde47952fcd None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:15.131123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aad01b-98b4-4c20-a533-48d9a6338221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:15.132564 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aad01b-98b4-4c20-a533-48d9a6338221 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:15.132901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aad01b-98b4-4c20-a533-48d9a6338221 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:15.133096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aad01b-98b4-4c20-a533-48d9a6338221 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:15.133957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aad01b-98b4-4c20-a533-48d9a6338221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:15.136312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aad01b-98b4-4c20-a533-48d9a6338221 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:15.144407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dd7af-ef2f-4118-89c7-838a4847f849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:15.146362 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dd7af-ef2f-4118-89c7-838a4847f849 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:15.157716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1102/2217] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:54:15.172840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1116/2218] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:15 2022] POST /identity/v3/projects => generated 357 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:15.174467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dd7af-ef2f-4118-89c7-838a4847f849 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:15.195689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dd7af-ef2f-4118-89c7-838a4847f849 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:15.236792 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:15.269921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42d1d-6516-4d68-aff0-5448eafa7ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:15.270596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42d1d-6516-4d68-aff0-5448eafa7ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:15.270874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42d1d-6516-4d68-aff0-5448eafa7ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:15.271152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42d1d-6516-4d68-aff0-5448eafa7ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:15.272443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42d1d-6516-4d68-aff0-5448eafa7ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:15.274581 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42d1d-6516-4d68-aff0-5448eafa7ae9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:15.297553 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1103/2219] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:15 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:15.306109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aad01b-98b4-4c20-a533-48d9a6338221 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:15.316347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aad01b-98b4-4c20-a533-48d9a6338221 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:15.331341 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469fdaf-b422-41e7-abad-6eeb19a4fa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:15.334800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469fdaf-b422-41e7-abad-6eeb19a4fa90 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:15.338963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469fdaf-b422-41e7-abad-6eeb19a4fa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:15.339253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469fdaf-b422-41e7-abad-6eeb19a4fa90 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:15.340023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469fdaf-b422-41e7-abad-6eeb19a4fa90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:15.342610 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469fdaf-b422-41e7-abad-6eeb19a4fa90 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:15.370636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1117/2220] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:15.372242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42d1d-6516-4d68-aff0-5448eafa7ae9 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:15.385159 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42d1d-6516-4d68-aff0-5448eafa7ae9 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:15.393038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f9e72-5376-45f4-a96e-d0f80f9ae31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:15.394623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f9e72-5376-45f4-a96e-d0f80f9ae31c admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:15.395050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f9e72-5376-45f4-a96e-d0f80f9ae31c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:15.395531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f9e72-5376-45f4-a96e-d0f80f9ae31c admin admin] PATH_INFO: `/v3/projects/21550a1ccf2d41a787e63fe82b336600/users/76583f4c363547ce8d32240d4d2bd6c0/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:15.417174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f9e72-5376-45f4-a96e-d0f80f9ae31c admin admin] RBAC: Authorizing `identity:create_grant(project_id=21550a1ccf2d41a787e63fe82b336600, user_id=76583f4c363547ce8d32240d4d2bd6c0, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:15.421067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f9e72-5376-45f4-a96e-d0f80f9ae31c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:15.489706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1104/2221] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:15 2022] PUT /identity/v3/projects/21550a1ccf2d41a787e63fe82b336600/users/76583f4c363547ce8d32240d4d2bd6c0/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:15.493102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469fdaf-b422-41e7-abad-6eeb19a4fa90 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:15.505120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469fdaf-b422-41e7-abad-6eeb19a4fa90 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:15.623752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e62979-b094-4d31-b83c-af6d7d062bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:15.624609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e62979-b094-4d31-b83c-af6d7d062bfc admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:15.625020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e62979-b094-4d31-b83c-af6d7d062bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:15.625370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e62979-b094-4d31-b83c-af6d7d062bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:15.626220 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e62979-b094-4d31-b83c-af6d7d062bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:15.627879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e62979-b094-4d31-b83c-af6d7d062bfc admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:15.639604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1118/2222] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:15.643566 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f9e72-5376-45f4-a96e-d0f80f9ae31c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:15.655000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f9e72-5376-45f4-a96e-d0f80f9ae31c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:15.660539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61829e37-0983-4f7f-a88d-d9a5c755dcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:15.661424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61829e37-0983-4f7f-a88d-d9a5c755dcd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:15.661742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61829e37-0983-4f7f-a88d-d9a5c755dcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:15.662016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61829e37-0983-4f7f-a88d-d9a5c755dcd3 admin admin] PATH_INFO: `/v3/projects/21550a1ccf2d41a787e63fe82b336600/users/76583f4c363547ce8d32240d4d2bd6c0/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:15.664479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61829e37-0983-4f7f-a88d-d9a5c755dcd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21550a1ccf2d41a787e63fe82b336600, user_id=76583f4c363547ce8d32240d4d2bd6c0, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:15.667180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61829e37-0983-4f7f-a88d-d9a5c755dcd3 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:15.678835 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1105/2223] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:15 2022] PUT /identity/v3/projects/21550a1ccf2d41a787e63fe82b336600/users/76583f4c363547ce8d32240d4d2bd6c0/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:15.690233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edee32-9705-4c18-8dd2-24c9bc9eb639 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:15.690664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edee32-9705-4c18-8dd2-24c9bc9eb639 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:15.691257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47edee32-9705-4c18-8dd2-24c9bc9eb639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:15.721317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-47edee32-9705-4c18-8dd2-24c9bc9eb639 None None] MFA Rules not processed for user `76583f4c363547ce8d32240d4d2bd6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:15.722937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-47edee32-9705-4c18-8dd2-24c9bc9eb639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:15.727183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:15.795664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1119/2224] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:15 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:15.808547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61829e37-0983-4f7f-a88d-d9a5c755dcd3 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:15.869842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61829e37-0983-4f7f-a88d-d9a5c755dcd3 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:15.874609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddd218-b328-4506-b468-1dd58b3dc6c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:15.875934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddd218-b328-4506-b468-1dd58b3dc6c8 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:15.875934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddd218-b328-4506-b468-1dd58b3dc6c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:15.875934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddd218-b328-4506-b468-1dd58b3dc6c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:15.890788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddd218-b328-4506-b468-1dd58b3dc6c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:15.892553 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddd218-b328-4506-b468-1dd58b3dc6c8 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:15.911833 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1106/2225] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:15 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:16.019360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47edee32-9705-4c18-8dd2-24c9bc9eb639 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:16.021041 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-47edee32-9705-4c18-8dd2-24c9bc9eb639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:16.034887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47edee32-9705-4c18-8dd2-24c9bc9eb639 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:16.039333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ac7a-db75-439e-a056-4a5cd41c8e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:16.039982 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ac7a-db75-439e-a056-4a5cd41c8e7f service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:16.040257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ac7a-db75-439e-a056-4a5cd41c8e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:16.040680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2ac7a-db75-439e-a056-4a5cd41c8e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:16.104446 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2ac7a-db75-439e-a056-4a5cd41c8e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:16.105964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2ac7a-db75-439e-a056-4a5cd41c8e7f service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:16.115240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1120/2226] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:16 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:25.899086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=56,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:26.110905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=55 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:38.893383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975dcd0-fd46-4c04-8c1c-c8abe2a44a12 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:38.893952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975dcd0-fd46-4c04-8c1c-c8abe2a44a12 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:38.894047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975dcd0-fd46-4c04-8c1c-c8abe2a44a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:38.914365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-b975dcd0-fd46-4c04-8c1c-c8abe2a44a12 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:38.916501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b975dcd0-fd46-4c04-8c1c-c8abe2a44a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:39.018267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1107/2227] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:54:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:39.077936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ac7a-db75-439e-a056-4a5cd41c8e7f service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:39.081605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-1be2ac7a-db75-439e-a056-4a5cd41c8e7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:39.210940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2ac7a-db75-439e-a056-4a5cd41c8e7f service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:39.218806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e0864-1363-47d5-a344-0a424ef88339 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:39.219902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e0864-1363-47d5-a344-0a424ef88339 service swift] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:39.220550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e0864-1363-47d5-a344-0a424ef88339 service swift] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:39.221010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e0864-1363-47d5-a344-0a424ef88339 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:39.233845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e0864-1363-47d5-a344-0a424ef88339 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:39.235831 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e0864-1363-47d5-a344-0a424ef88339 service swift] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:39.273840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1121/2228] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:54:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:54:39.471420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975dcd0-fd46-4c04-8c1c-c8abe2a44a12 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:39.625783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975dcd0-fd46-4c04-8c1c-c8abe2a44a12 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:39.633726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:39.634613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:39.634895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:39.635172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] PATH_INFO: `/v3/users/c048704fabf546bfa676bdc9524f3e75` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:39.636411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c048704fabf546bfa676bdc9524f3e75)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:39.638430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:39.770162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c048704fabf546bfa676bdc9524f3e75'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:39.793405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c048704fabf546bfa676bdc9524f3e75'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:39.810519 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c048704fabf546bfa676bdc9524f3e75'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:39.823108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c048704fabf546bfa676bdc9524f3e75'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:39.826276 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c048704fabf546bfa676bdc9524f3e75'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:39.841063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1108/2229] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:39 2022] DELETE /identity/v3/users/c048704fabf546bfa676bdc9524f3e75 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:39.854909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cab33d-f6e8-4879-9855-e265f7a283b3 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:39.855218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cab33d-f6e8-4879-9855-e265f7a283b3 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:39.855644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cab33d-f6e8-4879-9855-e265f7a283b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:39.890175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-46cab33d-f6e8-4879-9855-e265f7a283b3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:39.891440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-46cab33d-f6e8-4879-9855-e265f7a283b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:40.026703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1122/2230] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:40.036365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:40.038320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:40.074711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:40.089526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cab33d-f6e8-4879-9855-e265f7a283b3 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:40.091173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-46cab33d-f6e8-4879-9855-e265f7a283b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:40.108216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cab33d-f6e8-4879-9855-e265f7a283b3 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:40.115713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:40.117307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:40.118030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:40.118527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] PATH_INFO: `/v3/users/69e73e9ed6b548aa811cb01243be18e1` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:40.120107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e73e9ed6b548aa811cb01243be18e1)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:40.122267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:40.124124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f39aa-bd48-4e6e-bc84-638d5d7f1dd2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:40.128279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cb000-8244-4891-8f85-521b4d143c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:40.129621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cb000-8244-4891-8f85-521b4d143c8b service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:40.130982 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cb000-8244-4891-8f85-521b4d143c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:40.131880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cb000-8244-4891-8f85-521b4d143c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:40.147786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cb000-8244-4891-8f85-521b4d143c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:40.149418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cb000-8244-4891-8f85-521b4d143c8b service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:40.161549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1109/2231] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:40.224886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e73e9ed6b548aa811cb01243be18e1'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:40.238214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e73e9ed6b548aa811cb01243be18e1'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:40.241198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e73e9ed6b548aa811cb01243be18e1'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:40.258116 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e73e9ed6b548aa811cb01243be18e1'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:40.259163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e73e9ed6b548aa811cb01243be18e1'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:40.277714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1123/2232] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:40 2022] DELETE /identity/v3/users/69e73e9ed6b548aa811cb01243be18e1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:40.282612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59319bfa-b256-44c1-9ca7-4af7e66c3c3f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:40.283288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59319bfa-b256-44c1-9ca7-4af7e66c3c3f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:40.283966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59319bfa-b256-44c1-9ca7-4af7e66c3c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:40.306307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-59319bfa-b256-44c1-9ca7-4af7e66c3c3f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:40.307636 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-59319bfa-b256-44c1-9ca7-4af7e66c3c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:40.404770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1110/2233] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:40.415736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:40.417563 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:40.423792 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:40.606880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5abcf8-22dc-4627-aaea-554fa3f93f0f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:40.619533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78a6a8-79d0-4f20-be43-b5577c1a9043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:40.620396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78a6a8-79d0-4f20-be43-b5577c1a9043 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:40.620749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78a6a8-79d0-4f20-be43-b5577c1a9043 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:40.621125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78a6a8-79d0-4f20-be43-b5577c1a9043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:40.642641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59319bfa-b256-44c1-9ca7-4af7e66c3c3f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:40.643408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78a6a8-79d0-4f20-be43-b5577c1a9043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:40.644086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-59319bfa-b256-44c1-9ca7-4af7e66c3c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:40.646903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78a6a8-79d0-4f20-be43-b5577c1a9043 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:40.660630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59319bfa-b256-44c1-9ca7-4af7e66c3c3f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:40.671796 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1124/2234] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:40.679891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:40.680771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:40.681052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:40.681329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] PATH_INFO: `/v3/projects/3635a147c8214c38b96de99037249a13` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:40.688680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3635a147c8214c38b96de99037249a13)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:40.690689 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:40.748070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3635a147c8214c38b96de99037249a13', 'request_id': 'req-668a2c5f-cf62-4b81-9efe-6b8901535c33'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:40.758502 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3635a147c8214c38b96de99037249a13', 'request_id': 'req-668a2c5f-cf62-4b81-9efe-6b8901535c33'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:40.768539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3635a147c8214c38b96de99037249a13', 'request_id': 'req-668a2c5f-cf62-4b81-9efe-6b8901535c33'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:40.782947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] The token cache is being invalidate because project 3635a147c8214c38b96de99037249a13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:40.783407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:40.784115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-668a2c5f-cf62-4b81-9efe-6b8901535c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:40.785983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1111/2235] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:40 2022] DELETE /identity/v3/projects/3635a147c8214c38b96de99037249a13 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:40.795312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5f5fc-bd36-47c8-ac49-7810c526e974 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:40.796324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5f5fc-bd36-47c8-ac49-7810c526e974 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:40.796324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5f5fc-bd36-47c8-ac49-7810c526e974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:40.823432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-50d5f5fc-bd36-47c8-ac49-7810c526e974 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:40.825756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-50d5f5fc-bd36-47c8-ac49-7810c526e974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:40.908026 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1125/2236] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:40.984001 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf4730-702a-43a9-883b-95afd9df326a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:40.984387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf4730-702a-43a9-883b-95afd9df326a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:40.984761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf4730-702a-43a9-883b-95afd9df326a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.013829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7cdf4730-702a-43a9-883b-95afd9df326a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:41.015123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7cdf4730-702a-43a9-883b-95afd9df326a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:41.025256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1112/2237] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:41.033757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5f5fc-bd36-47c8-ac49-7810c526e974 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:41.045982 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5f5fc-bd36-47c8-ac49-7810c526e974 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:41.053480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2b4f5-bb09-4ffb-9b3f-ae4e5217e033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:41.054421 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2b4f5-bb09-4ffb-9b3f-ae4e5217e033 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:41.054781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2b4f5-bb09-4ffb-9b3f-ae4e5217e033 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:41.055786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2b4f5-bb09-4ffb-9b3f-ae4e5217e033 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.056669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2b4f5-bb09-4ffb-9b3f-ae4e5217e033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:54:41.057144 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2b4f5-bb09-4ffb-9b3f-ae4e5217e033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:41.059580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2b4f5-bb09-4ffb-9b3f-ae4e5217e033 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:41.080313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1126/2238] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:54:41.082561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf4730-702a-43a9-883b-95afd9df326a None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:41.104275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf4730-702a-43a9-883b-95afd9df326a None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:41.109829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c466163-d4e5-4104-a9fe-cf9f54ed528f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:41.110505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c466163-d4e5-4104-a9fe-cf9f54ed528f admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:41.110787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c466163-d4e5-4104-a9fe-cf9f54ed528f admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:41.111065 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c466163-d4e5-4104-a9fe-cf9f54ed528f admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.111767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c466163-d4e5-4104-a9fe-cf9f54ed528f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:41.113771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c466163-d4e5-4104-a9fe-cf9f54ed528f admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:41.166447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1113/2239] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:41 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:41.170684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2b4f5-bb09-4ffb-9b3f-ae4e5217e033 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:41.185191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2b4f5-bb09-4ffb-9b3f-ae4e5217e033 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:41.261863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be31884-df9a-4b0c-a01d-e8fbca8563ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:41.262546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be31884-df9a-4b0c-a01d-e8fbca8563ed admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:41.262832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be31884-df9a-4b0c-a01d-e8fbca8563ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:41.263109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be31884-df9a-4b0c-a01d-e8fbca8563ed admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.263996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be31884-df9a-4b0c-a01d-e8fbca8563ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:41.266033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be31884-df9a-4b0c-a01d-e8fbca8563ed admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:41.290610 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1127/2240] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:41 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:41.294629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c466163-d4e5-4104-a9fe-cf9f54ed528f admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:41.312943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c466163-d4e5-4104-a9fe-cf9f54ed528f admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:41.318881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd720a1a-0ba8-49cb-a796-08dda59d234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:41.319433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd720a1a-0ba8-49cb-a796-08dda59d234c admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:41.320375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd720a1a-0ba8-49cb-a796-08dda59d234c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:41.320375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd720a1a-0ba8-49cb-a796-08dda59d234c admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.320715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd720a1a-0ba8-49cb-a796-08dda59d234c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:41.322511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd720a1a-0ba8-49cb-a796-08dda59d234c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:41.338200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1114/2241] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:41.339173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be31884-df9a-4b0c-a01d-e8fbca8563ed admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:41.352530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be31884-df9a-4b0c-a01d-e8fbca8563ed admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:41.358019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5fca9-2d83-45c2-9b23-19f739374dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:41.358842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5fca9-2d83-45c2-9b23-19f739374dab admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:41.359200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5fca9-2d83-45c2-9b23-19f739374dab admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:41.359550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5fca9-2d83-45c2-9b23-19f739374dab admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.360431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5fca9-2d83-45c2-9b23-19f739374dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:41.362592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5fca9-2d83-45c2-9b23-19f739374dab admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:41.379260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1128/2242] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:41.381735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd720a1a-0ba8-49cb-a796-08dda59d234c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:41.395022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd720a1a-0ba8-49cb-a796-08dda59d234c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:41.406757 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2efad9-e883-4cdd-942a-79cb7f0fffb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:41.407931 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2efad9-e883-4cdd-942a-79cb7f0fffb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:41.408348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2efad9-e883-4cdd-942a-79cb7f0fffb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:41.408717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2efad9-e883-4cdd-942a-79cb7f0fffb0 admin admin] PATH_INFO: `/v3/projects/262e5b1a6f6248cca455437d23b8d441/users/80267c31f3fc4b3fbc8dc7ed03326eeb/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.418723 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:41.431103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2efad9-e883-4cdd-942a-79cb7f0fffb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=262e5b1a6f6248cca455437d23b8d441, user_id=80267c31f3fc4b3fbc8dc7ed03326eeb, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:41.434651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2efad9-e883-4cdd-942a-79cb7f0fffb0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:41.453263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1115/2243] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:41 2022] PUT /identity/v3/projects/262e5b1a6f6248cca455437d23b8d441/users/80267c31f3fc4b3fbc8dc7ed03326eeb/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:41.458173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ed2f0-f021-486f-b473-e1b3c7711c4e None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:41.459101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ed2f0-f021-486f-b473-e1b3c7711c4e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:41.459793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ed2f0-f021-486f-b473-e1b3c7711c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.480399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-dd3ed2f0-f021-486f-b473-e1b3c7711c4e None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:41.481830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-dd3ed2f0-f021-486f-b473-e1b3c7711c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:41.488329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:41.615661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1129/2244] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:41.622855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2efad9-e883-4cdd-942a-79cb7f0fffb0 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:41.626196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7b2efad9-e883-4cdd-942a-79cb7f0fffb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:41.642164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ed2f0-f021-486f-b473-e1b3c7711c4e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:41.644144 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-dd3ed2f0-f021-486f-b473-e1b3c7711c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:41.657466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2efad9-e883-4cdd-942a-79cb7f0fffb0 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:41.665980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:41.667175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:41.667563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:41.668167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] PATH_INFO: `/v3/projects/5e80831f3b804a9b8eecd37cb006f195` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.669487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e80831f3b804a9b8eecd37cb006f195)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:41.671966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:41.751613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e80831f3b804a9b8eecd37cb006f195', 'request_id': 'req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:41.779830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e80831f3b804a9b8eecd37cb006f195', 'request_id': 'req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:41.802290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e80831f3b804a9b8eecd37cb006f195', 'request_id': 'req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:41.814880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ed2f0-f021-486f-b473-e1b3c7711c4e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:41.822603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] The token cache is being invalidate because project 5e80831f3b804a9b8eecd37cb006f195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:41.822907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:41.823954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:41.827036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1116/2245] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:41 2022] DELETE /identity/v3/projects/5e80831f3b804a9b8eecd37cb006f195 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:41.831066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:41.833995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:41.930594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89ae50-198c-4622-aeee-533e77e90d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:41.931261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89ae50-198c-4622-aeee-533e77e90d50 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:41.931538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89ae50-198c-4622-aeee-533e77e90d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:41.931810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89ae50-198c-4622-aeee-533e77e90d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.936535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f7be2-6154-46cd-93e9-6a5f829b9d23 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:41.937322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4d89ae50-198c-4622-aeee-533e77e90d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:41.948722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:41.949920 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:41.950943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:41.951384 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] PATH_INFO: `/v3/users/76583f4c363547ce8d32240d4d2bd6c0` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:41.954755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89ae50-198c-4622-aeee-533e77e90d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:41.956576 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89ae50-198c-4622-aeee-533e77e90d50 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:41.961209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76583f4c363547ce8d32240d4d2bd6c0)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:41.963760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:41.973637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1130/2246] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:42.038025 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76583f4c363547ce8d32240d4d2bd6c0'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:42.055087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76583f4c363547ce8d32240d4d2bd6c0'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:42.063669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76583f4c363547ce8d32240d4d2bd6c0'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:42.065388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:42.095223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76583f4c363547ce8d32240d4d2bd6c0'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:42.096394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-87a2bdf5-5065-4b10-a4e7-1d397d62dec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76583f4c363547ce8d32240d4d2bd6c0'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:42.121721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1117/2247] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:41 2022] DELETE /identity/v3/users/76583f4c363547ce8d32240d4d2bd6c0 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:43.315229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89ae50-198c-4622-aeee-533e77e90d50 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:43.328342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89ae50-198c-4622-aeee-533e77e90d50 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:43.468427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:43.469448 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:43.470504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:43.470790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] PATH_INFO: `/v3/projects/21550a1ccf2d41a787e63fe82b336600` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:43.476469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21550a1ccf2d41a787e63fe82b336600)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:43.479214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:43.515475 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:43.581846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21550a1ccf2d41a787e63fe82b336600', 'request_id': 'req-c161836b-f194-426c-a586-58cfe529ede8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:43.619168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21550a1ccf2d41a787e63fe82b336600', 'request_id': 'req-c161836b-f194-426c-a586-58cfe529ede8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:43.635238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21550a1ccf2d41a787e63fe82b336600', 'request_id': 'req-c161836b-f194-426c-a586-58cfe529ede8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:43.691069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] The token cache is being invalidate because project 21550a1ccf2d41a787e63fe82b336600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:43.691069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:43.700695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-c161836b-f194-426c-a586-58cfe529ede8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:43.702813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1131/2248] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:43 2022] DELETE /identity/v3/projects/21550a1ccf2d41a787e63fe82b336600 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:43.707419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd003c2-d482-4d73-a609-228bb0ed01a5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:43.707973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd003c2-d482-4d73-a609-228bb0ed01a5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:43.707973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd003c2-d482-4d73-a609-228bb0ed01a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:43.735283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ddd003c2-d482-4d73-a609-228bb0ed01a5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:43.736559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ddd003c2-d482-4d73-a609-228bb0ed01a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:43.877668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1118/2249] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:43.886691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a428-2da5-4a5f-9da5-4e9a6f1ea091 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:43.886969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a428-2da5-4a5f-9da5-4e9a6f1ea091 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:43.893883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a428-2da5-4a5f-9da5-4e9a6f1ea091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:43.933982 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-e9d5a428-2da5-4a5f-9da5-4e9a6f1ea091 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:43.935513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e9d5a428-2da5-4a5f-9da5-4e9a6f1ea091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:43.955496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1132/2250] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:43.960861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd003c2-d482-4d73-a609-228bb0ed01a5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:43.986885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd003c2-d482-4d73-a609-228bb0ed01a5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:44.003535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8556b-7da8-4b04-83a2-ec22a1b69c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:44.004479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8556b-7da8-4b04-83a2-ec22a1b69c14 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:44.004995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8556b-7da8-4b04-83a2-ec22a1b69c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:44.005317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8556b-7da8-4b04-83a2-ec22a1b69c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:44.006093 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8556b-7da8-4b04-83a2-ec22a1b69c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:54:44.006401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8556b-7da8-4b04-83a2-ec22a1b69c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:44.008006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8556b-7da8-4b04-83a2-ec22a1b69c14 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:44.027820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a428-2da5-4a5f-9da5-4e9a6f1ea091 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:44.038178 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1119/2251] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:54:44.042115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a428-2da5-4a5f-9da5-4e9a6f1ea091 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:44.052579 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1e2ca-bd33-46e2-87bb-4204e16b2985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:44.053516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1e2ca-bd33-46e2-87bb-4204e16b2985 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:44.053997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1e2ca-bd33-46e2-87bb-4204e16b2985 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:44.054448 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1e2ca-bd33-46e2-87bb-4204e16b2985 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:44.055439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1e2ca-bd33-46e2-87bb-4204e16b2985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:44.060104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1e2ca-bd33-46e2-87bb-4204e16b2985 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:44.124417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1133/2252] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:44 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:44.128680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8556b-7da8-4b04-83a2-ec22a1b69c14 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:44.143940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8556b-7da8-4b04-83a2-ec22a1b69c14 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:44.276814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3e4a9-bd70-40ce-a8ae-dab6b92b9968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:44.277518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3e4a9-bd70-40ce-a8ae-dab6b92b9968 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:44.277518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3e4a9-bd70-40ce-a8ae-dab6b92b9968 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:44.277518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3e4a9-bd70-40ce-a8ae-dab6b92b9968 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:44.277518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3e4a9-bd70-40ce-a8ae-dab6b92b9968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:44.285468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3e4a9-bd70-40ce-a8ae-dab6b92b9968 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:44.340736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1120/2253] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:44 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:44.342388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1e2ca-bd33-46e2-87bb-4204e16b2985 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:44.365962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1e2ca-bd33-46e2-87bb-4204e16b2985 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:44.372052 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70c27b-d0c7-47b9-b1f2-c3d1d2ca8683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:44.378098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70c27b-d0c7-47b9-b1f2-c3d1d2ca8683 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:44.378485 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70c27b-d0c7-47b9-b1f2-c3d1d2ca8683 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:44.379316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70c27b-d0c7-47b9-b1f2-c3d1d2ca8683 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:44.383830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70c27b-d0c7-47b9-b1f2-c3d1d2ca8683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:44.385675 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70c27b-d0c7-47b9-b1f2-c3d1d2ca8683 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:44.419077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1134/2254] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:44.423380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3e4a9-bd70-40ce-a8ae-dab6b92b9968 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:44.437809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3e4a9-bd70-40ce-a8ae-dab6b92b9968 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:44.454281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5b809-9b8d-4a8f-980a-36fb9949de87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:44.455044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5b809-9b8d-4a8f-980a-36fb9949de87 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:44.455399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5b809-9b8d-4a8f-980a-36fb9949de87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:44.455741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5b809-9b8d-4a8f-980a-36fb9949de87 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:44.456709 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5b809-9b8d-4a8f-980a-36fb9949de87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:44.459402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5b809-9b8d-4a8f-980a-36fb9949de87 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:44.496206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1121/2255] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:44.504335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70c27b-d0c7-47b9-b1f2-c3d1d2ca8683 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:44.518797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70c27b-d0c7-47b9-b1f2-c3d1d2ca8683 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:44.528177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943b148-e6e1-4b21-979e-df34ee9acc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:44.529425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943b148-e6e1-4b21-979e-df34ee9acc2f admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:44.529811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943b148-e6e1-4b21-979e-df34ee9acc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:44.530199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943b148-e6e1-4b21-979e-df34ee9acc2f admin admin] PATH_INFO: `/v3/projects/e3369aca34d0441ab2683b7050637844/users/3c467bbcf0c242e5abed5bf18913edd2/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:44.557739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943b148-e6e1-4b21-979e-df34ee9acc2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3369aca34d0441ab2683b7050637844, user_id=3c467bbcf0c242e5abed5bf18913edd2, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:44.564056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943b148-e6e1-4b21-979e-df34ee9acc2f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:44.645681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1135/2256] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:44 2022] PUT /identity/v3/projects/e3369aca34d0441ab2683b7050637844/users/3c467bbcf0c242e5abed5bf18913edd2/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:44.650545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc28c9f-1272-4950-8e8b-6b78e70a4dff None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:44.651581 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc28c9f-1272-4950-8e8b-6b78e70a4dff None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:44.652109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc28c9f-1272-4950-8e8b-6b78e70a4dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:44.721070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-1cc28c9f-1272-4950-8e8b-6b78e70a4dff None None] MFA Rules not processed for user `3c467bbcf0c242e5abed5bf18913edd2`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:44.722542 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1cc28c9f-1272-4950-8e8b-6b78e70a4dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:44.802967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:44.860763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1122/2257] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:44.866196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943b148-e6e1-4b21-979e-df34ee9acc2f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:44.991960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943b148-e6e1-4b21-979e-df34ee9acc2f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:45.007207 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c32c4a-ba53-4c4d-89f3-55523943b54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:45.007992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32c4a-ba53-4c4d-89f3-55523943b54f admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:45.008335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32c4a-ba53-4c4d-89f3-55523943b54f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:45.008682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c32c4a-ba53-4c4d-89f3-55523943b54f admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:45.009466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c32c4a-ba53-4c4d-89f3-55523943b54f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:45.012678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c32c4a-ba53-4c4d-89f3-55523943b54f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:45.074423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1136/2258] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:44 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:45.078618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc28c9f-1272-4950-8e8b-6b78e70a4dff None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:45.100310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc28c9f-1272-4950-8e8b-6b78e70a4dff None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:45.250001 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e016631-0d94-48ce-968d-2fc0b24089ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:45.250368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e016631-0d94-48ce-968d-2fc0b24089ef admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:45.250368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e016631-0d94-48ce-968d-2fc0b24089ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:45.250741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e016631-0d94-48ce-968d-2fc0b24089ef admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:45.251395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e016631-0d94-48ce-968d-2fc0b24089ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:45.253399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e016631-0d94-48ce-968d-2fc0b24089ef admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:45.290631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c32c4a-ba53-4c4d-89f3-55523943b54f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:45.304615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c32c4a-ba53-4c4d-89f3-55523943b54f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:45.306462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1123/2259] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:45 2022] POST /identity/v3/users => generated 290 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:45.311840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc620591-738c-4087-8361-e220f3eddc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:45.312529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc620591-738c-4087-8361-e220f3eddc31 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:45.312812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc620591-738c-4087-8361-e220f3eddc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:45.313099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc620591-738c-4087-8361-e220f3eddc31 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:45.314144 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc620591-738c-4087-8361-e220f3eddc31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:45.316293 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc620591-738c-4087-8361-e220f3eddc31 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:45.332638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1137/2260] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:45.337347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e016631-0d94-48ce-968d-2fc0b24089ef admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:45.349842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e016631-0d94-48ce-968d-2fc0b24089ef admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:45.358144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e7050-c531-4d1e-9e91-c03648f3e527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:45.358950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e7050-c531-4d1e-9e91-c03648f3e527 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:45.359119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e7050-c531-4d1e-9e91-c03648f3e527 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:45.359946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e7050-c531-4d1e-9e91-c03648f3e527 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:45.361243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e7050-c531-4d1e-9e91-c03648f3e527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:45.362999 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e7050-c531-4d1e-9e91-c03648f3e527 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:45.384932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1124/2261] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:45.389096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc620591-738c-4087-8361-e220f3eddc31 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:45.400437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc620591-738c-4087-8361-e220f3eddc31 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:45.405842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7afbe5-3e6a-4e53-aecd-763e81985dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:45.406751 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7afbe5-3e6a-4e53-aecd-763e81985dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:45.407035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7afbe5-3e6a-4e53-aecd-763e81985dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:45.407310 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7afbe5-3e6a-4e53-aecd-763e81985dc1 admin admin] PATH_INFO: `/v3/projects/707da67256a54abcaeadccd64a5898db/users/44a508b034bf4709ac4b63a3b7dee5a7/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:45.420700 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:45.425186 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7afbe5-3e6a-4e53-aecd-763e81985dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=707da67256a54abcaeadccd64a5898db, user_id=44a508b034bf4709ac4b63a3b7dee5a7, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:45.429141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7afbe5-3e6a-4e53-aecd-763e81985dc1 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:45.459467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1138/2262] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:45 2022] PUT /identity/v3/projects/707da67256a54abcaeadccd64a5898db/users/44a508b034bf4709ac4b63a3b7dee5a7/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:45.465989 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82697c-08c6-43e6-a0af-c7b445acf230 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:45.466370 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82697c-08c6-43e6-a0af-c7b445acf230 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:45.466717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82697c-08c6-43e6-a0af-c7b445acf230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:45.511989 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9d82697c-08c6-43e6-a0af-c7b445acf230 None None] MFA Rules not processed for user `44a508b034bf4709ac4b63a3b7dee5a7`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:45.513696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9d82697c-08c6-43e6-a0af-c7b445acf230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:45.650560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7afbe5-3e6a-4e53-aecd-763e81985dc1 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:45.670372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1125/2263] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:45.767212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7afbe5-3e6a-4e53-aecd-763e81985dc1 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:45.790050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8fb90-ed0e-4dbb-8ee6-541bacd9a110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:45.790740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8fb90-ed0e-4dbb-8ee6-541bacd9a110 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:45.791019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8fb90-ed0e-4dbb-8ee6-541bacd9a110 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:45.791290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8fb90-ed0e-4dbb-8ee6-541bacd9a110 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:45.791994 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8fb90-ed0e-4dbb-8ee6-541bacd9a110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:45.793980 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8fb90-ed0e-4dbb-8ee6-541bacd9a110 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:45.855046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1139/2264] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:45 2022] POST /identity/v3/projects => generated 379 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:45.858872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82697c-08c6-43e6-a0af-c7b445acf230 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:45.888950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82697c-08c6-43e6-a0af-c7b445acf230 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:45.963002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:46.000863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5c290-fad9-4d5a-8c9a-d16b39b1debe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:46.001807 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5c290-fad9-4d5a-8c9a-d16b39b1debe admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:46.002710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5c290-fad9-4d5a-8c9a-d16b39b1debe admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:46.003170 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5c290-fad9-4d5a-8c9a-d16b39b1debe admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:46.004003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5c290-fad9-4d5a-8c9a-d16b39b1debe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:46.008446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5c290-fad9-4d5a-8c9a-d16b39b1debe admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:46.063271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8fb90-ed0e-4dbb-8ee6-541bacd9a110 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:46.079543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8fb90-ed0e-4dbb-8ee6-541bacd9a110 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:46.085734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1126/2265] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:45 2022] POST /identity/v3/users => generated 296 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:46.093055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba07683-bf2b-4418-991d-257caccec495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:46.094187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba07683-bf2b-4418-991d-257caccec495 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:46.094710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba07683-bf2b-4418-991d-257caccec495 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:46.095830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba07683-bf2b-4418-991d-257caccec495 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:46.096689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba07683-bf2b-4418-991d-257caccec495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:46.100976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba07683-bf2b-4418-991d-257caccec495 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:46.134522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5c290-fad9-4d5a-8c9a-d16b39b1debe admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:46.144489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1140/2266] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:54:46.147273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5c290-fad9-4d5a-8c9a-d16b39b1debe admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:46.161360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941706c0-ab6a-4c73-b312-5abb9ec9b0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:46.162297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941706c0-ab6a-4c73-b312-5abb9ec9b0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:46.162579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941706c0-ab6a-4c73-b312-5abb9ec9b0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:46.162858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941706c0-ab6a-4c73-b312-5abb9ec9b0d2 admin admin] PATH_INFO: `/v3/projects/b23d19a0cd9945e48b61f8662498fc53/users/6780a4bcaad64b6cbde465cf4fbb780b/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:46.199938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941706c0-ab6a-4c73-b312-5abb9ec9b0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23d19a0cd9945e48b61f8662498fc53, user_id=6780a4bcaad64b6cbde465cf4fbb780b, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:46.206302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941706c0-ab6a-4c73-b312-5abb9ec9b0d2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:46.250550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba07683-bf2b-4418-991d-257caccec495 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:46.251116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1127/2267] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:46 2022] PUT /identity/v3/projects/b23d19a0cd9945e48b61f8662498fc53/users/6780a4bcaad64b6cbde465cf4fbb780b/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:46.276966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba07683-bf2b-4418-991d-257caccec495 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:46.439313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307ddce-6a26-457f-a08b-51b8c42b481e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:46.440129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307ddce-6a26-457f-a08b-51b8c42b481e admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:46.440382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307ddce-6a26-457f-a08b-51b8c42b481e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:46.440658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307ddce-6a26-457f-a08b-51b8c42b481e admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:46.441399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307ddce-6a26-457f-a08b-51b8c42b481e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:46.443965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307ddce-6a26-457f-a08b-51b8c42b481e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:46.476442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941706c0-ab6a-4c73-b312-5abb9ec9b0d2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:46.493339 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941706c0-ab6a-4c73-b312-5abb9ec9b0d2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:46.493797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1141/2268] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:46.510135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b82df-9f50-45db-91be-71e8248b105e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:46.511223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b82df-9f50-45db-91be-71e8248b105e admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:46.511515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b82df-9f50-45db-91be-71e8248b105e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:46.511792 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b82df-9f50-45db-91be-71e8248b105e admin admin] PATH_INFO: `/v3/projects/b23d19a0cd9945e48b61f8662498fc53/users/6780a4bcaad64b6cbde465cf4fbb780b/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:46.520409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b82df-9f50-45db-91be-71e8248b105e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b23d19a0cd9945e48b61f8662498fc53, user_id=6780a4bcaad64b6cbde465cf4fbb780b, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:46.532905 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b82df-9f50-45db-91be-71e8248b105e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:46.575695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1128/2269] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:46 2022] PUT /identity/v3/projects/b23d19a0cd9945e48b61f8662498fc53/users/6780a4bcaad64b6cbde465cf4fbb780b/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:46.586735 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743354b-bd47-40e7-a819-d9a779d10d33 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:46.587066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743354b-bd47-40e7-a819-d9a779d10d33 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:46.587299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743354b-bd47-40e7-a819-d9a779d10d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:46.644387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7743354b-bd47-40e7-a819-d9a779d10d33 None None] MFA Rules not processed for user `6780a4bcaad64b6cbde465cf4fbb780b`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:46.646638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7743354b-bd47-40e7-a819-d9a779d10d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:46.762024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:46.814889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1142/2270] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:46.828967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b82df-9f50-45db-91be-71e8248b105e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:46.831042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2c4b82df-9f50-45db-91be-71e8248b105e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:46.968699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b82df-9f50-45db-91be-71e8248b105e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:46.986500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2c194-aea8-49ef-8090-6d027ed18d0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:46.987177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2c194-aea8-49ef-8090-6d027ed18d0e service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:46.987463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2c194-aea8-49ef-8090-6d027ed18d0e service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:46.987741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2c194-aea8-49ef-8090-6d027ed18d0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:47.010774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2c194-aea8-49ef-8090-6d027ed18d0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:47.012246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2c194-aea8-49ef-8090-6d027ed18d0e service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:47.035829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1129/2271] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:47.123744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743354b-bd47-40e7-a819-d9a779d10d33 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:47.144107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743354b-bd47-40e7-a819-d9a779d10d33 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:47.155845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36abfef-42e3-454b-b486-fd99fa01bb92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:47.156904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36abfef-42e3-454b-b486-fd99fa01bb92 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:47.157201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36abfef-42e3-454b-b486-fd99fa01bb92 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:47.157529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36abfef-42e3-454b-b486-fd99fa01bb92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:47.336281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36abfef-42e3-454b-b486-fd99fa01bb92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:47.337797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36abfef-42e3-454b-b486-fd99fa01bb92 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:47.367775 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1143/2272] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:50.340234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f9af6-7b91-4dd7-8c60-91b7810cc8ba None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:50.340823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f9af6-7b91-4dd7-8c60-91b7810cc8ba None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:50.340823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f9af6-7b91-4dd7-8c60-91b7810cc8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:50.390123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-0f1f9af6-7b91-4dd7-8c60-91b7810cc8ba None None] MFA Rules not processed for user `80267c31f3fc4b3fbc8dc7ed03326eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:50.391357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0f1f9af6-7b91-4dd7-8c60-91b7810cc8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:50.546087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1130/2273] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:50.550591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36abfef-42e3-454b-b486-fd99fa01bb92 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:50.557427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f36abfef-42e3-454b-b486-fd99fa01bb92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:50.643966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36abfef-42e3-454b-b486-fd99fa01bb92 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:50.657663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd675429-7dcf-41a0-bc42-1d7f52f77d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:50.658700 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd675429-7dcf-41a0-bc42-1d7f52f77d26 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:50.659174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd675429-7dcf-41a0-bc42-1d7f52f77d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:50.659801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd675429-7dcf-41a0-bc42-1d7f52f77d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:50.660645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd675429-7dcf-41a0-bc42-1d7f52f77d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:50.663818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd675429-7dcf-41a0-bc42-1d7f52f77d26 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:50.732212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1144/2274] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:54:50 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:50.740437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f9af6-7b91-4dd7-8c60-91b7810cc8ba None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:50.763302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f9af6-7b91-4dd7-8c60-91b7810cc8ba None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:50.849252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:50.926997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1034af-7aa2-4799-9bf0-c56445acdb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:50.928810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1034af-7aa2-4799-9bf0-c56445acdb5d admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:50.929480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1034af-7aa2-4799-9bf0-c56445acdb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:50.929899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1034af-7aa2-4799-9bf0-c56445acdb5d admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:50.933632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1034af-7aa2-4799-9bf0-c56445acdb5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:50.935816 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1034af-7aa2-4799-9bf0-c56445acdb5d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:51.000343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1131/2275] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:54:50 2022] POST /identity/v3/users => generated 295 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:54:51.002122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd675429-7dcf-41a0-bc42-1d7f52f77d26 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:51.018138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd675429-7dcf-41a0-bc42-1d7f52f77d26 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:51.042888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3a7a8-e1e3-4462-a5a5-0c83b8fb22b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:51.043773 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3a7a8-e1e3-4462-a5a5-0c83b8fb22b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:51.044302 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3a7a8-e1e3-4462-a5a5-0c83b8fb22b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:51.044705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3a7a8-e1e3-4462-a5a5-0c83b8fb22b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:51.045854 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3a7a8-e1e3-4462-a5a5-0c83b8fb22b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:51.047664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3a7a8-e1e3-4462-a5a5-0c83b8fb22b0 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:51.087834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1034af-7aa2-4799-9bf0-c56445acdb5d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:51.093194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1145/2276] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:51.094781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3a7a8-e1e3-4462-a5a5-0c83b8fb22b0 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:51.102580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1034af-7aa2-4799-9bf0-c56445acdb5d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:51.124448 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3a7a8-e1e3-4462-a5a5-0c83b8fb22b0 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:51.126087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:51.127486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:51.128218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:51.129106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] PATH_INFO: `/v3/users/3c467bbcf0c242e5abed5bf18913edd2` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:51.130545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c467bbcf0c242e5abed5bf18913edd2)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:51.132685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:51.136709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42edec1-c98e-45cd-b4c1-d5ce1c97748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:51.137949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42edec1-c98e-45cd-b4c1-d5ce1c97748b admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:51.138407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42edec1-c98e-45cd-b4c1-d5ce1c97748b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:51.138859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42edec1-c98e-45cd-b4c1-d5ce1c97748b admin admin] PATH_INFO: `/v3/projects/bdfabef2ed454d6590af05df16f55610/users/b771b0dd79db41c4ab6ab0ce9037bac9/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:51.166771 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42edec1-c98e-45cd-b4c1-d5ce1c97748b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfabef2ed454d6590af05df16f55610, user_id=b771b0dd79db41c4ab6ab0ce9037bac9, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:51.169644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42edec1-c98e-45cd-b4c1-d5ce1c97748b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:51.223466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1146/2277] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:51 2022] PUT /identity/v3/projects/bdfabef2ed454d6590af05df16f55610/users/b771b0dd79db41c4ab6ab0ce9037bac9/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:51.228383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42edec1-c98e-45cd-b4c1-d5ce1c97748b admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:51.257196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c467bbcf0c242e5abed5bf18913edd2'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:51.278425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c467bbcf0c242e5abed5bf18913edd2'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:51.291562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c467bbcf0c242e5abed5bf18913edd2'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:51.300009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:51.321173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c467bbcf0c242e5abed5bf18913edd2'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:51.325132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-c7bfc33b-27cd-4fb6-a344-5d1fc951289a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c467bbcf0c242e5abed5bf18913edd2'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:51.349266 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1132/2278] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:51 2022] DELETE /identity/v3/users/3c467bbcf0c242e5abed5bf18913edd2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:54:51.355889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5584d-4b70-48a3-b651-8491e570f37c None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:51.356163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5584d-4b70-48a3-b651-8491e570f37c None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:51.357340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5584d-4b70-48a3-b651-8491e570f37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:51.422098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42edec1-c98e-45cd-b4c1-d5ce1c97748b admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:51.428058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-03b5584d-4b70-48a3-b651-8491e570f37c None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:51.430684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-03b5584d-4b70-48a3-b651-8491e570f37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:51.556909 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89763ef8-50ee-4302-a0a2-87dcf391125a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:51.557846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763ef8-50ee-4302-a0a2-87dcf391125a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:51.558237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763ef8-50ee-4302-a0a2-87dcf391125a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:51.558553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763ef8-50ee-4302-a0a2-87dcf391125a admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:51.559327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89763ef8-50ee-4302-a0a2-87dcf391125a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:51.563510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89763ef8-50ee-4302-a0a2-87dcf391125a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:51.584195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1147/2279] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:54:51.588088 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89763ef8-50ee-4302-a0a2-87dcf391125a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:51.590519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-89763ef8-50ee-4302-a0a2-87dcf391125a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:51.619358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89763ef8-50ee-4302-a0a2-87dcf391125a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:51.625311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1133/2280] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:51.651823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5584d-4b70-48a3-b651-8491e570f37c None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:51.653347 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-03b5584d-4b70-48a3-b651-8491e570f37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:51.658211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eefa8-85d9-41dc-8842-d4e1a5b33acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:51.667891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eefa8-85d9-41dc-8842-d4e1a5b33acf admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:51.668451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eefa8-85d9-41dc-8842-d4e1a5b33acf admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:51.668957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eefa8-85d9-41dc-8842-d4e1a5b33acf admin admin] PATH_INFO: `/v3/projects/bdfabef2ed454d6590af05df16f55610/users/b771b0dd79db41c4ab6ab0ce9037bac9/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:51.673533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45eefa8-85d9-41dc-8842-d4e1a5b33acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfabef2ed454d6590af05df16f55610, user_id=b771b0dd79db41c4ab6ab0ce9037bac9, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:51.681797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45eefa8-85d9-41dc-8842-d4e1a5b33acf admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:51.722913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1148/2281] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:54:51 2022] PUT /identity/v3/projects/bdfabef2ed454d6590af05df16f55610/users/b771b0dd79db41c4ab6ab0ce9037bac9/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:51.771660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:51.798726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5584d-4b70-48a3-b651-8491e570f37c None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:51.937605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0649bd4-9743-4616-8b45-b3b700713eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:51.938376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0649bd4-9743-4616-8b45-b3b700713eed service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:51.938727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0649bd4-9743-4616-8b45-b3b700713eed service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:51.939080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0649bd4-9743-4616-8b45-b3b700713eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:52.076672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0649bd4-9743-4616-8b45-b3b700713eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:52.078609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0649bd4-9743-4616-8b45-b3b700713eed service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:52.116307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1134/2282] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:54:52.737513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eefa8-85d9-41dc-8842-d4e1a5b33acf admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:52.739632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c45eefa8-85d9-41dc-8842-d4e1a5b33acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:52.768301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eefa8-85d9-41dc-8842-d4e1a5b33acf admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:52.785951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:52.786102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:52.786470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:52.786949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] PATH_INFO: `/v3/projects/e3369aca34d0441ab2683b7050637844` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:52.788282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3369aca34d0441ab2683b7050637844)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:52.790312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:52.899829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3369aca34d0441ab2683b7050637844', 'request_id': 'req-f4e42505-433a-4975-adcd-32f9bbe30776'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:52.916760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3369aca34d0441ab2683b7050637844', 'request_id': 'req-f4e42505-433a-4975-adcd-32f9bbe30776'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:52.924901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3369aca34d0441ab2683b7050637844', 'request_id': 'req-f4e42505-433a-4975-adcd-32f9bbe30776'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:52.950871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] The token cache is being invalidate because project e3369aca34d0441ab2683b7050637844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:52.951167 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:52.951935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-f4e42505-433a-4975-adcd-32f9bbe30776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:52.953784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1149/2283] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:52 2022] DELETE /identity/v3/projects/e3369aca34d0441ab2683b7050637844 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:52.959569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0649bd4-9743-4616-8b45-b3b700713eed service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:52.992400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0649bd4-9743-4616-8b45-b3b700713eed service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:53.140586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:53.142314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:53.142877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:53.143081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] PATH_INFO: `/v3/users/44a508b034bf4709ac4b63a3b7dee5a7` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:53.160124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44a508b034bf4709ac4b63a3b7dee5a7)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:53.164242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:53.340553 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44a508b034bf4709ac4b63a3b7dee5a7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:53.362485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:53.427595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44a508b034bf4709ac4b63a3b7dee5a7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:53.464500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44a508b034bf4709ac4b63a3b7dee5a7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:53.481795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44a508b034bf4709ac4b63a3b7dee5a7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:53.483587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44a508b034bf4709ac4b63a3b7dee5a7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:53.526367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1135/2284] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:52 2022] DELETE /identity/v3/users/44a508b034bf4709ac4b63a3b7dee5a7 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:54.627216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba457c-d002-4770-84da-c2b984fcd724 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:54.628087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba457c-d002-4770-84da-c2b984fcd724 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:54.628535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba457c-d002-4770-84da-c2b984fcd724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:54.666710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5fba457c-d002-4770-84da-c2b984fcd724 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:54:54.668269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5fba457c-d002-4770-84da-c2b984fcd724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:54.778474 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:54.782730 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:54.854881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:54.914160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1150/2285] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:54:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:54:54.929803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba457c-d002-4770-84da-c2b984fcd724 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:54.931553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5fba457c-d002-4770-84da-c2b984fcd724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:54.974219 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0566f54-b708-4ef0-8ec2-ea69117583c1 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:54.998741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:55.000649 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:55.000942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:55.001226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] PATH_INFO: `/v3/projects/707da67256a54abcaeadccd64a5898db` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:55.022402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=707da67256a54abcaeadccd64a5898db)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:55.032148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:55.125253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba457c-d002-4770-84da-c2b984fcd724 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:55.134183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2189ff-99bf-4a21-a11f-84798d19d9ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:55.134855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2189ff-99bf-4a21-a11f-84798d19d9ae service swift] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:55.135134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2189ff-99bf-4a21-a11f-84798d19d9ae service swift] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:55.135404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2189ff-99bf-4a21-a11f-84798d19d9ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:55.163045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2189ff-99bf-4a21-a11f-84798d19d9ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:55.164731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2189ff-99bf-4a21-a11f-84798d19d9ae service swift] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:55.196543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1151/2286] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:54:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:54:55.201991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '707da67256a54abcaeadccd64a5898db', 'request_id': 'req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:55.227753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '707da67256a54abcaeadccd64a5898db', 'request_id': 'req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:55.244245 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '707da67256a54abcaeadccd64a5898db', 'request_id': 'req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:55.281745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] The token cache is being invalidate because project 707da67256a54abcaeadccd64a5898db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:54:55.282257 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:55.283095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:55.288007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1136/2287] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:54:54 2022] DELETE /identity/v3/projects/707da67256a54abcaeadccd64a5898db => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:55.292174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2189ff-99bf-4a21-a11f-84798d19d9ae service swift] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:55.310074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2189ff-99bf-4a21-a11f-84798d19d9ae service swift] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:55.517722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:55.522642 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:55.523198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:55.523699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] PATH_INFO: `/v3/users/6780a4bcaad64b6cbde465cf4fbb780b` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:55.534062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6780a4bcaad64b6cbde465cf4fbb780b)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:55.537334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:55.671793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6780a4bcaad64b6cbde465cf4fbb780b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:55.688282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6780a4bcaad64b6cbde465cf4fbb780b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:55.690035 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6780a4bcaad64b6cbde465cf4fbb780b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:55.727793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6780a4bcaad64b6cbde465cf4fbb780b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:55.729229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6780a4bcaad64b6cbde465cf4fbb780b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:54:55.755973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:55.768624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1152/2288] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:54:55 2022] DELETE /identity/v3/users/6780a4bcaad64b6cbde465cf4fbb780b => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:54:57.316094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:54:57.318872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:57.626505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec35d7-b3fa-4b09-87f9-2a85d842e0c9 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:54:57.651703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:54:57.759972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa1274-8c52-41d4-a923-46549f2637c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:54:57.760795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa1274-8c52-41d4-a923-46549f2637c6 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:54:57.761185 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa1274-8c52-41d4-a923-46549f2637c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:54:57.761561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa1274-8c52-41d4-a923-46549f2637c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:54:57.768941 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0daa1274-8c52-41d4-a923-46549f2637c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:54:57.996879 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa1274-8c52-41d4-a923-46549f2637c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:54:58.000555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa1274-8c52-41d4-a923-46549f2637c6 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:54:58.058096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1137/2289] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:54:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 741 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:55:00.361870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:00.364272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:00.628558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75a18-f2de-4314-b39c-fff2b62eb01b admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:00.648685 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:00.649618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:00.649915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:00.650195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] PATH_INFO: `/v3/projects/b23d19a0cd9945e48b61f8662498fc53` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:00.652374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b23d19a0cd9945e48b61f8662498fc53)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:00.663453 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:00.801384 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b23d19a0cd9945e48b61f8662498fc53', 'request_id': 'req-7dbe191c-7c14-40c0-bf9c-ac455bec7809'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:00.834541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b23d19a0cd9945e48b61f8662498fc53', 'request_id': 'req-7dbe191c-7c14-40c0-bf9c-ac455bec7809'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:00.845543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b23d19a0cd9945e48b61f8662498fc53', 'request_id': 'req-7dbe191c-7c14-40c0-bf9c-ac455bec7809'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:00.880707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] The token cache is being invalidate because project b23d19a0cd9945e48b61f8662498fc53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:55:00.881015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:00.884541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7dbe191c-7c14-40c0-bf9c-ac455bec7809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:00.887264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1153/2290] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:55:00 2022] DELETE /identity/v3/projects/b23d19a0cd9945e48b61f8662498fc53 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:00.896929 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ecb51-86aa-447c-83dd-2fb2104a5d82 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:00.897338 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ecb51-86aa-447c-83dd-2fb2104a5d82 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:00.898124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ecb51-86aa-447c-83dd-2fb2104a5d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:00.928097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-125ecb51-86aa-447c-83dd-2fb2104a5d82 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:00.929549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-125ecb51-86aa-447c-83dd-2fb2104a5d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:01.000295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772f798-918e-4392-8239-10d7a6d27f0c None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:01.001716 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772f798-918e-4392-8239-10d7a6d27f0c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:01.001716 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772f798-918e-4392-8239-10d7a6d27f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:01.048195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c772f798-918e-4392-8239-10d7a6d27f0c None None] MFA Rules not processed for user `0267350339c44000bcc49906cfa8f0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:01.051877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c772f798-918e-4392-8239-10d7a6d27f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:01.133729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1138/2291] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:01.147716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35065e1e-a486-4a7b-8de8-9eeb170c9fba None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:01.148275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35065e1e-a486-4a7b-8de8-9eeb170c9fba None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:01.148849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35065e1e-a486-4a7b-8de8-9eeb170c9fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:01.191311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-35065e1e-a486-4a7b-8de8-9eeb170c9fba None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:01.196939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-35065e1e-a486-4a7b-8de8-9eeb170c9fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:01.209712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1154/2292] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:55:01.214283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1139/2293] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:55:01.220467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772f798-918e-4392-8239-10d7a6d27f0c None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:01.234147 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772f798-918e-4392-8239-10d7a6d27f0c None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:01.242103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b80cb-2ae1-41e1-bb45-769b18041ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:01.243862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b80cb-2ae1-41e1-bb45-769b18041ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:01.244517 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b80cb-2ae1-41e1-bb45-769b18041ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:01.245050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b80cb-2ae1-41e1-bb45-769b18041ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:01.246051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b80cb-2ae1-41e1-bb45-769b18041ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:55:01.248744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b80cb-2ae1-41e1-bb45-769b18041ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:01.250990 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b80cb-2ae1-41e1-bb45-769b18041ed1 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:01.287560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35065e1e-a486-4a7b-8de8-9eeb170c9fba None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:01.293227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1155/2294] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:55:01.299735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35065e1e-a486-4a7b-8de8-9eeb170c9fba None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:01.317033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdf0dc-2bbb-440b-b211-7f91b75af1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:01.318281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdf0dc-2bbb-440b-b211-7f91b75af1ca admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:01.318585 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdf0dc-2bbb-440b-b211-7f91b75af1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:01.318864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdf0dc-2bbb-440b-b211-7f91b75af1ca admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:01.319606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdf0dc-2bbb-440b-b211-7f91b75af1ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:01.321756 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdf0dc-2bbb-440b-b211-7f91b75af1ca admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:01.391367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1140/2295] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:55:01 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:55:01.396429 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b80cb-2ae1-41e1-bb45-769b18041ed1 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:01.421039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b80cb-2ae1-41e1-bb45-769b18041ed1 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:01.460286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:01.612887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bff9d9-e6b0-47ed-a357-61c1722de966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:01.615094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bff9d9-e6b0-47ed-a357-61c1722de966 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:01.615094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bff9d9-e6b0-47ed-a357-61c1722de966 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:01.615299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bff9d9-e6b0-47ed-a357-61c1722de966 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:01.623062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bff9d9-e6b0-47ed-a357-61c1722de966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:01.628717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bff9d9-e6b0-47ed-a357-61c1722de966 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:01.690893 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1156/2296] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:55:01 2022] POST /identity/v3/users => generated 292 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:55:01.693486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdf0dc-2bbb-440b-b211-7f91b75af1ca admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:01.709394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdf0dc-2bbb-440b-b211-7f91b75af1ca admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:01.733068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047f217-fa1f-431a-a5f7-fcc8378d2e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:01.733801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047f217-fa1f-431a-a5f7-fcc8378d2e20 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:01.734087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047f217-fa1f-431a-a5f7-fcc8378d2e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:01.734370 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047f217-fa1f-431a-a5f7-fcc8378d2e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:01.735121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047f217-fa1f-431a-a5f7-fcc8378d2e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:01.736791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047f217-fa1f-431a-a5f7-fcc8378d2e20 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:01.770968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1141/2297] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:01.774232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bff9d9-e6b0-47ed-a357-61c1722de966 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:01.810783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bff9d9-e6b0-47ed-a357-61c1722de966 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:01.858658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1293e6f-5e8c-4de5-9773-be1deb4d9705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:01.859363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1293e6f-5e8c-4de5-9773-be1deb4d9705 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:01.859642 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1293e6f-5e8c-4de5-9773-be1deb4d9705 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:01.859919 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1293e6f-5e8c-4de5-9773-be1deb4d9705 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:01.860698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1293e6f-5e8c-4de5-9773-be1deb4d9705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:01.872549 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1293e6f-5e8c-4de5-9773-be1deb4d9705 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:01.897104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1157/2298] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:01.901611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047f217-fa1f-431a-a5f7-fcc8378d2e20 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:01.916040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047f217-fa1f-431a-a5f7-fcc8378d2e20 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:01.924061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d8fe4-161d-4e4b-bcf6-f4c9bd129077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:01.927958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d8fe4-161d-4e4b-bcf6-f4c9bd129077 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:01.928870 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d8fe4-161d-4e4b-bcf6-f4c9bd129077 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:01.929230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d8fe4-161d-4e4b-bcf6-f4c9bd129077 admin admin] PATH_INFO: `/v3/projects/63e559f7e171454c93a9261f85e31fe1/users/c989d9b7750d4053be19e89fb15b157d/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:01.949860 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d8fe4-161d-4e4b-bcf6-f4c9bd129077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e559f7e171454c93a9261f85e31fe1, user_id=c989d9b7750d4053be19e89fb15b157d, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:01.952650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d8fe4-161d-4e4b-bcf6-f4c9bd129077 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:01.981697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:01.988162 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1142/2299] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:55:01 2022] PUT /identity/v3/projects/63e559f7e171454c93a9261f85e31fe1/users/c989d9b7750d4053be19e89fb15b157d/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:55:02.004218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cefa8-4ff9-4b0a-9a89-3018b4f6f648 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:02.004627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cefa8-4ff9-4b0a-9a89-3018b4f6f648 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:02.004985 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cefa8-4ff9-4b0a-9a89-3018b4f6f648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:02.063432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-182cefa8-4ff9-4b0a-9a89-3018b4f6f648 None None] MFA Rules not processed for user `c989d9b7750d4053be19e89fb15b157d`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:02.068008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-182cefa8-4ff9-4b0a-9a89-3018b4f6f648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:02.239057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1158/2300] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:02.246416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d8fe4-161d-4e4b-bcf6-f4c9bd129077 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:02.346246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d7235-57d5-495f-8e1b-35bf775e9100 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:02.346589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d7235-57d5-495f-8e1b-35bf775e9100 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:02.346865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d7235-57d5-495f-8e1b-35bf775e9100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:02.424121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d8fe4-161d-4e4b-bcf6-f4c9bd129077 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:02.438625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203803b8-e30a-4398-bdc3-641146e820d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:02.439233 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203803b8-e30a-4398-bdc3-641146e820d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:02.439526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203803b8-e30a-4398-bdc3-641146e820d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:02.439810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203803b8-e30a-4398-bdc3-641146e820d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:02.440635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203803b8-e30a-4398-bdc3-641146e820d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:02.444337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203803b8-e30a-4398-bdc3-641146e820d1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:02.445297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-682d7235-57d5-495f-8e1b-35bf775e9100 None None] MFA Rules not processed for user `b771b0dd79db41c4ab6ab0ce9037bac9`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:02.450122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-682d7235-57d5-495f-8e1b-35bf775e9100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:02.534798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1143/2301] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:55:02 2022] POST /identity/v3/projects => generated 383 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:55:02.537422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203803b8-e30a-4398-bdc3-641146e820d1 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:02.699608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1159/2302] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:55:02.744919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203803b8-e30a-4398-bdc3-641146e820d1 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:02.769037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de483ff3-b931-43cc-a507-1a5026ccf0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:02.770491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483ff3-b931-43cc-a507-1a5026ccf0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:02.770637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483ff3-b931-43cc-a507-1a5026ccf0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:02.770845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de483ff3-b931-43cc-a507-1a5026ccf0f5 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:02.771565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de483ff3-b931-43cc-a507-1a5026ccf0f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:02.773529 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de483ff3-b931-43cc-a507-1a5026ccf0f5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:02.809845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d7235-57d5-495f-8e1b-35bf775e9100 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:02.811496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-682d7235-57d5-495f-8e1b-35bf775e9100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:02.817522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1144/2303] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:55:02 2022] POST /identity/v3/users => generated 298 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:55:02.821336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de483ff3-b931-43cc-a507-1a5026ccf0f5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:02.845270 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de483ff3-b931-43cc-a507-1a5026ccf0f5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:02.850645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:02.865193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485a59d-3582-4f5d-a3c6-f1f266f4e8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:02.865942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485a59d-3582-4f5d-a3c6-f1f266f4e8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:02.867000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485a59d-3582-4f5d-a3c6-f1f266f4e8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:02.870686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485a59d-3582-4f5d-a3c6-f1f266f4e8c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:02.871475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485a59d-3582-4f5d-a3c6-f1f266f4e8c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:02.878070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485a59d-3582-4f5d-a3c6-f1f266f4e8c5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:02.926505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1145/2304] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:02.928953 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485a59d-3582-4f5d-a3c6-f1f266f4e8c5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:02.948133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485a59d-3582-4f5d-a3c6-f1f266f4e8c5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:02.963447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b7be0-a5ce-44aa-b316-fd64e247e564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:02.964280 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b7be0-a5ce-44aa-b316-fd64e247e564 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:02.964606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b7be0-a5ce-44aa-b316-fd64e247e564 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:02.964889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b7be0-a5ce-44aa-b316-fd64e247e564 admin admin] PATH_INFO: `/v3/projects/c42453a43ee04faeab0618ffc2cd9381/users/378d587fa19645e38e4bb7d8a11c8c0d/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:02.998068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b7be0-a5ce-44aa-b316-fd64e247e564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42453a43ee04faeab0618ffc2cd9381, user_id=378d587fa19645e38e4bb7d8a11c8c0d, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:03.000782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b7be0-a5ce-44aa-b316-fd64e247e564 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:03.031307 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1146/2305] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:55:02 2022] PUT /identity/v3/projects/c42453a43ee04faeab0618ffc2cd9381/users/378d587fa19645e38e4bb7d8a11c8c0d/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:03.035044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b7be0-a5ce-44aa-b316-fd64e247e564 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:03.039573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d7235-57d5-495f-8e1b-35bf775e9100 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:03.127519 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:03.163190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b7be0-a5ce-44aa-b316-fd64e247e564 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:03.174060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7fad2-caf5-4a11-a1bc-f6f57c934297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:03.174426 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f343bd3-a2a1-4438-ae80-08fbde2ff992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:03.175534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7fad2-caf5-4a11-a1bc-f6f57c934297 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:03.176100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7fad2-caf5-4a11-a1bc-f6f57c934297 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:03.176469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f343bd3-a2a1-4438-ae80-08fbde2ff992 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:03.176717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7fad2-caf5-4a11-a1bc-f6f57c934297 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:03.178357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f343bd3-a2a1-4438-ae80-08fbde2ff992 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:03.178671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f343bd3-a2a1-4438-ae80-08fbde2ff992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:03.179989 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7fad2-caf5-4a11-a1bc-f6f57c934297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:03.187019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8f343bd3-a2a1-4438-ae80-08fbde2ff992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:03.189825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7fad2-caf5-4a11-a1bc-f6f57c934297 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:03.222350 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1147/2306] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:55:03.224882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7fad2-caf5-4a11-a1bc-f6f57c934297 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:03.250439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7fad2-caf5-4a11-a1bc-f6f57c934297 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:03.258490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2304c42-b24e-4809-ad45-ec00cbcb9d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:03.276694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2304c42-b24e-4809-ad45-ec00cbcb9d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:03.277176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2304c42-b24e-4809-ad45-ec00cbcb9d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:03.277718 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2304c42-b24e-4809-ad45-ec00cbcb9d98 admin admin] PATH_INFO: `/v3/projects/c42453a43ee04faeab0618ffc2cd9381/users/378d587fa19645e38e4bb7d8a11c8c0d/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:03.294983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2304c42-b24e-4809-ad45-ec00cbcb9d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42453a43ee04faeab0618ffc2cd9381, user_id=378d587fa19645e38e4bb7d8a11c8c0d, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:03.304473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2304c42-b24e-4809-ad45-ec00cbcb9d98 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:03.352024 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1148/2307] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:55:03 2022] PUT /identity/v3/projects/c42453a43ee04faeab0618ffc2cd9381/users/378d587fa19645e38e4bb7d8a11c8c0d/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:03.355589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be859ce1-4fae-4908-a4a8-9b09dd6a23ac None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:03.356150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be859ce1-4fae-4908-a4a8-9b09dd6a23ac None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:03.356606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be859ce1-4fae-4908-a4a8-9b09dd6a23ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:03.368638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f343bd3-a2a1-4438-ae80-08fbde2ff992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:03.372076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f343bd3-a2a1-4438-ae80-08fbde2ff992 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:03.400257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1160/2308] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:03.406432 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-be859ce1-4fae-4908-a4a8-9b09dd6a23ac None None] MFA Rules not processed for user `378d587fa19645e38e4bb7d8a11c8c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:03.408206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-be859ce1-4fae-4908-a4a8-9b09dd6a23ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:03.680880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1149/2309] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:03.720495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f343bd3-a2a1-4438-ae80-08fbde2ff992 service nova] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:03.723790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8f343bd3-a2a1-4438-ae80-08fbde2ff992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:03.860371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:03.871729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f343bd3-a2a1-4438-ae80-08fbde2ff992 service nova] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:04.027100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187ad3d-7da0-4c92-b92c-a6abe899ea61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:04.029223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ad3d-7da0-4c92-b92c-a6abe899ea61 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:04.032829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ad3d-7da0-4c92-b92c-a6abe899ea61 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:04.037991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ad3d-7da0-4c92-b92c-a6abe899ea61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:04.101540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187ad3d-7da0-4c92-b92c-a6abe899ea61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:04.115589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187ad3d-7da0-4c92-b92c-a6abe899ea61 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:04.166870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1161/2310] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:04.392713 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be859ce1-4fae-4908-a4a8-9b09dd6a23ac None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:04.404212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be859ce1-4fae-4908-a4a8-9b09dd6a23ac None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:04.420049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ccdbd-a536-4e52-b007-7c5b89bb44e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:04.420885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ccdbd-a536-4e52-b007-7c5b89bb44e7 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:04.421304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ccdbd-a536-4e52-b007-7c5b89bb44e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:04.421764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ccdbd-a536-4e52-b007-7c5b89bb44e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:04.695692 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ccdbd-a536-4e52-b007-7c5b89bb44e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:04.697181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ccdbd-a536-4e52-b007-7c5b89bb44e7 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:04.752145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1150/2311] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:05.690966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187ad3d-7da0-4c92-b92c-a6abe899ea61 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:05.870160 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187ad3d-7da0-4c92-b92c-a6abe899ea61 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:05.882381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:05.884189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:05.884189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:05.885805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] PATH_INFO: `/v3/users/c989d9b7750d4053be19e89fb15b157d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:05.887117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c989d9b7750d4053be19e89fb15b157d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:05.892789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:06.109383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c989d9b7750d4053be19e89fb15b157d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:06.139322 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c989d9b7750d4053be19e89fb15b157d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:06.140633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c989d9b7750d4053be19e89fb15b157d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:06.178798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c989d9b7750d4053be19e89fb15b157d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:06.182071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c989d9b7750d4053be19e89fb15b157d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:06.242490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1162/2312] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:55:05 2022] DELETE /identity/v3/users/c989d9b7750d4053be19e89fb15b157d => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:06.247278 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ea291-312f-4fc0-bf43-19f2296b1361 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:06.248578 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ea291-312f-4fc0-bf43-19f2296b1361 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:06.249624 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ea291-312f-4fc0-bf43-19f2296b1361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:06.321224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-043ea291-312f-4fc0-bf43-19f2296b1361 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:06.334181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-043ea291-312f-4fc0-bf43-19f2296b1361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:06.519121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:06.579164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1151/2313] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:06.615345 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:06.616977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:06.710744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:06.819242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7eaff-d21e-44aa-8b20-beb7e88f7742 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:06.836647 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529646c-9933-4bd7-8377-55e89e309f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:06.838031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529646c-9933-4bd7-8377-55e89e309f2a service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:06.838573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529646c-9933-4bd7-8377-55e89e309f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:06.838856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529646c-9933-4bd7-8377-55e89e309f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:06.882483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529646c-9933-4bd7-8377-55e89e309f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:06.884251 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529646c-9933-4bd7-8377-55e89e309f2a service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:06.962113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1163/2314] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:08.855483 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ea291-312f-4fc0-bf43-19f2296b1361 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:08.857517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-043ea291-312f-4fc0-bf43-19f2296b1361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:08.887869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ea291-312f-4fc0-bf43-19f2296b1361 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:08.910823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:08.911762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:08.912109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:08.912509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] PATH_INFO: `/v3/projects/63e559f7e171454c93a9261f85e31fe1` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:08.921276 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e559f7e171454c93a9261f85e31fe1)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:08.924158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:09.029138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e559f7e171454c93a9261f85e31fe1', 'request_id': 'req-d4fb2a60-0496-41e7-a4ff-c614da63bff9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:09.049279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e559f7e171454c93a9261f85e31fe1', 'request_id': 'req-d4fb2a60-0496-41e7-a4ff-c614da63bff9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:09.057192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e559f7e171454c93a9261f85e31fe1', 'request_id': 'req-d4fb2a60-0496-41e7-a4ff-c614da63bff9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:09.080448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] The token cache is being invalidate because project 63e559f7e171454c93a9261f85e31fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:55:09.080688 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:09.082156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:09.088503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1152/2315] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:55:08 2022] DELETE /identity/v3/projects/63e559f7e171454c93a9261f85e31fe1 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:09.089760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529646c-9933-4bd7-8377-55e89e309f2a service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:09.102668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529646c-9933-4bd7-8377-55e89e309f2a service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:09.224325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:09.225397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:09.229893 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:09.230295 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] PATH_INFO: `/v3/users/378d587fa19645e38e4bb7d8a11c8c0d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:09.233451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=378d587fa19645e38e4bb7d8a11c8c0d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:09.235503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:09.342628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '378d587fa19645e38e4bb7d8a11c8c0d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:09.365883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '378d587fa19645e38e4bb7d8a11c8c0d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:09.366941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '378d587fa19645e38e4bb7d8a11c8c0d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:09.402586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '378d587fa19645e38e4bb7d8a11c8c0d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:09.404136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8c0632da-c0be-481f-a4d7-9ece72a56b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '378d587fa19645e38e4bb7d8a11c8c0d'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:09.438069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1164/2316] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:55:09 2022] DELETE /identity/v3/users/378d587fa19645e38e4bb7d8a11c8c0d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:10.338345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:10.340386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:10.365763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb2a60-0496-41e7-a4ff-c614da63bff9 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:10.513735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:10.514940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:10.517134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:10.517620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] PATH_INFO: `/v3/projects/c42453a43ee04faeab0618ffc2cd9381` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:10.521679 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c42453a43ee04faeab0618ffc2cd9381)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:10.523867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:10.644016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c42453a43ee04faeab0618ffc2cd9381', 'request_id': 'req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:10.664488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c42453a43ee04faeab0618ffc2cd9381', 'request_id': 'req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:10.677461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c42453a43ee04faeab0618ffc2cd9381', 'request_id': 'req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:10.695174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:10.715565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] The token cache is being invalidate because project c42453a43ee04faeab0618ffc2cd9381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:55:10.715864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:10.718818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2f3e88a-3d7b-40c8-9bce-64895acbe47b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:10.724356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1153/2317] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:55:10 2022] DELETE /identity/v3/projects/c42453a43ee04faeab0618ffc2cd9381 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:10.729813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a41d15-9cbf-427f-bf7a-13af1205e5ff None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:10.730515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a41d15-9cbf-427f-bf7a-13af1205e5ff None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:10.730515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a41d15-9cbf-427f-bf7a-13af1205e5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:10.759807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-12a41d15-9cbf-427f-bf7a-13af1205e5ff None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:10.761046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-12a41d15-9cbf-427f-bf7a-13af1205e5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:10.856392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:10.909778 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d77a8-b0f1-4b36-b326-a742386ab222 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:10.910099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d77a8-b0f1-4b36-b326-a742386ab222 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:10.911116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d77a8-b0f1-4b36-b326-a742386ab222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:10.914143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1165/2318] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:11.056911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f63d77a8-b0f1-4b36-b326-a742386ab222 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:11.061791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f63d77a8-b0f1-4b36-b326-a742386ab222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:11.200361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a41d15-9cbf-427f-bf7a-13af1205e5ff None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:11.204850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1154/2319] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:11.360055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a41d15-9cbf-427f-bf7a-13af1205e5ff None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:11.406045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b3398-413e-406c-970e-064481373397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:11.407480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b3398-413e-406c-970e-064481373397 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:11.407937 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b3398-413e-406c-970e-064481373397 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:11.408373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b3398-413e-406c-970e-064481373397 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:11.409280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b3398-413e-406c-970e-064481373397 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:55:11.409780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b3398-413e-406c-970e-064481373397 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:11.414498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b3398-413e-406c-970e-064481373397 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:11.480316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1166/2320] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:55:11.486997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d77a8-b0f1-4b36-b326-a742386ab222 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:11.513396 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d77a8-b0f1-4b36-b326-a742386ab222 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:11.534858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db21ae9-79f4-4085-88ae-36589c484c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:11.534858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db21ae9-79f4-4085-88ae-36589c484c82 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:11.535815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db21ae9-79f4-4085-88ae-36589c484c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:11.542878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db21ae9-79f4-4085-88ae-36589c484c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:11.542878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db21ae9-79f4-4085-88ae-36589c484c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:11.585527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db21ae9-79f4-4085-88ae-36589c484c82 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:11.751971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1155/2321] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:55:11 2022] POST /identity/v3/projects => generated 375 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:55:11.752109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b3398-413e-406c-970e-064481373397 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:11.752109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b3398-413e-406c-970e-064481373397 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:11.994924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393df332-2cd0-48a8-a062-cef0a00add88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:11.995902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df332-2cd0-48a8-a062-cef0a00add88 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:11.996346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df332-2cd0-48a8-a062-cef0a00add88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:11.996779 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df332-2cd0-48a8-a062-cef0a00add88 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:11.997681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393df332-2cd0-48a8-a062-cef0a00add88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:12.000110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393df332-2cd0-48a8-a062-cef0a00add88 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:12.032859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1167/2322] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:55:11 2022] POST /identity/v3/users => generated 288 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:55:12.034214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db21ae9-79f4-4085-88ae-36589c484c82 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:12.047142 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db21ae9-79f4-4085-88ae-36589c484c82 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:12.051811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1979339-6e32-4f9a-905f-61d707d00d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:12.052680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1979339-6e32-4f9a-905f-61d707d00d55 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:12.052960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1979339-6e32-4f9a-905f-61d707d00d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:12.053236 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1979339-6e32-4f9a-905f-61d707d00d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:12.054040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1979339-6e32-4f9a-905f-61d707d00d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:12.055777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1979339-6e32-4f9a-905f-61d707d00d55 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:12.118371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1156/2323] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:12.119971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393df332-2cd0-48a8-a062-cef0a00add88 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:12.134736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393df332-2cd0-48a8-a062-cef0a00add88 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:12.139463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1f111-cdc2-4292-9da8-1886609ac6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:12.140151 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1f111-cdc2-4292-9da8-1886609ac6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:12.140422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1f111-cdc2-4292-9da8-1886609ac6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:12.140699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1f111-cdc2-4292-9da8-1886609ac6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:12.141553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1f111-cdc2-4292-9da8-1886609ac6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:12.143371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1f111-cdc2-4292-9da8-1886609ac6d0 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:12.158855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1168/2324] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:12.164252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1979339-6e32-4f9a-905f-61d707d00d55 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:12.180485 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1979339-6e32-4f9a-905f-61d707d00d55 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:12.200401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e77cf-ffe8-4769-b5ca-8b970326bf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:12.202403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e77cf-ffe8-4769-b5ca-8b970326bf76 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:12.202403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e77cf-ffe8-4769-b5ca-8b970326bf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:12.202696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e77cf-ffe8-4769-b5ca-8b970326bf76 admin admin] PATH_INFO: `/v3/projects/13acfdc03c10433aac900052a1355e78/users/5086331995784770b4615804398691c6/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:12.244574 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e77cf-ffe8-4769-b5ca-8b970326bf76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13acfdc03c10433aac900052a1355e78, user_id=5086331995784770b4615804398691c6, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:12.248232 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e77cf-ffe8-4769-b5ca-8b970326bf76 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:12.285926 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1157/2325] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:55:12 2022] PUT /identity/v3/projects/13acfdc03c10433aac900052a1355e78/users/5086331995784770b4615804398691c6/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:55:12.301101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e331c-34ca-462e-9b65-1750f58df663 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:12.301443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e331c-34ca-462e-9b65-1750f58df663 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:12.305966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e331c-34ca-462e-9b65-1750f58df663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:12.342183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-912e331c-34ca-462e-9b65-1750f58df663 None None] MFA Rules not processed for user `5086331995784770b4615804398691c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:12.343780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-912e331c-34ca-462e-9b65-1750f58df663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:12.487439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1169/2326] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:12.494638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e77cf-ffe8-4769-b5ca-8b970326bf76 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:12.682530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e77cf-ffe8-4769-b5ca-8b970326bf76 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:12.689443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e5dd4-4bc6-44df-88d1-d37e435fb2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:12.690608 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e5dd4-4bc6-44df-88d1-d37e435fb2c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:12.691265 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e5dd4-4bc6-44df-88d1-d37e435fb2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:12.691837 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e5dd4-4bc6-44df-88d1-d37e435fb2c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:12.692747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e5dd4-4bc6-44df-88d1-d37e435fb2c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:12.694996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e5dd4-4bc6-44df-88d1-d37e435fb2c2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:12.721780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:12.755690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1158/2327] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:55:12 2022] POST /identity/v3/projects => generated 375 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:55:12.759876 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e331c-34ca-462e-9b65-1750f58df663 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:12.772234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:12.777318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e331c-34ca-462e-9b65-1750f58df663 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:12.943342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a178bd9-4eea-494d-bd80-fa363ebda33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:12.944649 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a178bd9-4eea-494d-bd80-fa363ebda33a admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:12.944992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a178bd9-4eea-494d-bd80-fa363ebda33a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:12.945279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a178bd9-4eea-494d-bd80-fa363ebda33a admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:12.946247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a178bd9-4eea-494d-bd80-fa363ebda33a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:12.948317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a178bd9-4eea-494d-bd80-fa363ebda33a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:12.992658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1170/2328] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:55:12 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:55:12.993963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e5dd4-4bc6-44df-88d1-d37e435fb2c2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:13.008243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e5dd4-4bc6-44df-88d1-d37e435fb2c2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:13.030804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ddd5d-5dc0-49e6-9eeb-a3a2395afadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:13.031486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ddd5d-5dc0-49e6-9eeb-a3a2395afadf admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:13.031790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ddd5d-5dc0-49e6-9eeb-a3a2395afadf admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:13.032055 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ddd5d-5dc0-49e6-9eeb-a3a2395afadf admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:13.032845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ddd5d-5dc0-49e6-9eeb-a3a2395afadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:13.034488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ddd5d-5dc0-49e6-9eeb-a3a2395afadf admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:13.053626 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1159/2329] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:13.056204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a178bd9-4eea-494d-bd80-fa363ebda33a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:13.076028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a178bd9-4eea-494d-bd80-fa363ebda33a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:13.090780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07e467-993c-4a2d-adb8-230493203fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:13.091755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07e467-993c-4a2d-adb8-230493203fff admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:13.099750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07e467-993c-4a2d-adb8-230493203fff admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:13.100221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07e467-993c-4a2d-adb8-230493203fff admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:13.101179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07e467-993c-4a2d-adb8-230493203fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:13.107998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07e467-993c-4a2d-adb8-230493203fff admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:13.146164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1171/2330] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:13.156066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ddd5d-5dc0-49e6-9eeb-a3a2395afadf admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:13.179640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ddd5d-5dc0-49e6-9eeb-a3a2395afadf admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:13.185639 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa13c5e-e2b0-4c4f-a51c-2b025b476b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:13.186616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa13c5e-e2b0-4c4f-a51c-2b025b476b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:13.187052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa13c5e-e2b0-4c4f-a51c-2b025b476b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:13.187484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa13c5e-e2b0-4c4f-a51c-2b025b476b50 admin admin] PATH_INFO: `/v3/projects/730d7caf0db24ceaa194788743542408/users/e7f342dc594d4bdbb5a75500829b8074/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:13.208461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa13c5e-e2b0-4c4f-a51c-2b025b476b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=730d7caf0db24ceaa194788743542408, user_id=e7f342dc594d4bdbb5a75500829b8074, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:13.211414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa13c5e-e2b0-4c4f-a51c-2b025b476b50 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:13.239121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1160/2331] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:55:13 2022] PUT /identity/v3/projects/730d7caf0db24ceaa194788743542408/users/e7f342dc594d4bdbb5a75500829b8074/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:13.253279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a896aa-ca3b-40a0-b41e-399410dfb399 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:13.254520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a896aa-ca3b-40a0-b41e-399410dfb399 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:13.254799 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a896aa-ca3b-40a0-b41e-399410dfb399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:13.289394 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f3a896aa-ca3b-40a0-b41e-399410dfb399 None None] MFA Rules not processed for user `e7f342dc594d4bdbb5a75500829b8074`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:13.290666 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f3a896aa-ca3b-40a0-b41e-399410dfb399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:13.421672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1172/2332] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:13.426086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa13c5e-e2b0-4c4f-a51c-2b025b476b50 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:13.564223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa13c5e-e2b0-4c4f-a51c-2b025b476b50 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:13.580647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d9cb2-e950-485a-956e-697de8f9102b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:13.583324 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9cb2-e950-485a-956e-697de8f9102b admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:13.584172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9cb2-e950-485a-956e-697de8f9102b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:13.584555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9cb2-e950-485a-956e-697de8f9102b admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:13.585970 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d9cb2-e950-485a-956e-697de8f9102b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:13.589848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d9cb2-e950-485a-956e-697de8f9102b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:13.646278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a896aa-ca3b-40a0-b41e-399410dfb399 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:13.647893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1161/2333] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:55:13 2022] POST /identity/v3/projects => generated 375 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:55:13.662590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a896aa-ca3b-40a0-b41e-399410dfb399 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:13.796468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e60bc0-239e-473f-9fdc-be3284c0fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:13.797202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e60bc0-239e-473f-9fdc-be3284c0fa48 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:13.797528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e60bc0-239e-473f-9fdc-be3284c0fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:13.797875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e60bc0-239e-473f-9fdc-be3284c0fa48 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:13.799548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e60bc0-239e-473f-9fdc-be3284c0fa48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:13.801030 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e60bc0-239e-473f-9fdc-be3284c0fa48 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:13.856665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1173/2334] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:55:13 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:55:13.860696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d9cb2-e950-485a-956e-697de8f9102b admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:13.875234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d9cb2-e950-485a-956e-697de8f9102b admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:13.887818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e3133-ce7b-4c2c-99f9-da07b1c1159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:13.888676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e3133-ce7b-4c2c-99f9-da07b1c1159c admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:13.889064 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e3133-ce7b-4c2c-99f9-da07b1c1159c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:13.889720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e3133-ce7b-4c2c-99f9-da07b1c1159c admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:13.890950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e3133-ce7b-4c2c-99f9-da07b1c1159c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:13.894796 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e3133-ce7b-4c2c-99f9-da07b1c1159c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:13.920405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1162/2335] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:13.921897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e60bc0-239e-473f-9fdc-be3284c0fa48 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:13.933432 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:13.943603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e60bc0-239e-473f-9fdc-be3284c0fa48 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:13.949639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2f0fd-27fa-4a56-b8f5-5d4464c45a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:13.950678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2f0fd-27fa-4a56-b8f5-5d4464c45a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:13.950960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2f0fd-27fa-4a56-b8f5-5d4464c45a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:13.951229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2f0fd-27fa-4a56-b8f5-5d4464c45a06 admin admin] PATH_INFO: `/v3/projects/20d02d6730944ed4b81cc57c4ed051fc/users/0ff22fcd20c54ea1924f4a0588b2a94b/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:13.997109 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2f0fd-27fa-4a56-b8f5-5d4464c45a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20d02d6730944ed4b81cc57c4ed051fc, user_id=0ff22fcd20c54ea1924f4a0588b2a94b, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:14.000255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2f0fd-27fa-4a56-b8f5-5d4464c45a06 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:14.053510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1174/2336] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:55:13 2022] PUT /identity/v3/projects/20d02d6730944ed4b81cc57c4ed051fc/users/0ff22fcd20c54ea1924f4a0588b2a94b/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:14.058556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e3133-ce7b-4c2c-99f9-da07b1c1159c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:14.074255 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e3133-ce7b-4c2c-99f9-da07b1c1159c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:14.207397 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a327b4-6597-492b-a513-beef920f5794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:14.208132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a327b4-6597-492b-a513-beef920f5794 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:14.208464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a327b4-6597-492b-a513-beef920f5794 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:14.208775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a327b4-6597-492b-a513-beef920f5794 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:14.209556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a327b4-6597-492b-a513-beef920f5794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:14.211381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a327b4-6597-492b-a513-beef920f5794 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:14.231408 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1163/2337] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:14.239400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2f0fd-27fa-4a56-b8f5-5d4464c45a06 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:14.255113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2f0fd-27fa-4a56-b8f5-5d4464c45a06 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:14.270220 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aade79-5c28-4e2f-8bb5-1a2456810ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:14.272208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aade79-5c28-4e2f-8bb5-1a2456810ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:14.272799 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aade79-5c28-4e2f-8bb5-1a2456810ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:14.273458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aade79-5c28-4e2f-8bb5-1a2456810ee5 admin admin] PATH_INFO: `/v3/projects/20d02d6730944ed4b81cc57c4ed051fc/users/0ff22fcd20c54ea1924f4a0588b2a94b/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:14.280168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aade79-5c28-4e2f-8bb5-1a2456810ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20d02d6730944ed4b81cc57c4ed051fc, user_id=0ff22fcd20c54ea1924f4a0588b2a94b, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:14.284637 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aade79-5c28-4e2f-8bb5-1a2456810ee5 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:14.321619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1175/2338] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:55:14 2022] PUT /identity/v3/projects/20d02d6730944ed4b81cc57c4ed051fc/users/0ff22fcd20c54ea1924f4a0588b2a94b/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:55:14.332327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6672cc5-2340-4262-aeaa-00384ba0e11e None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:14.333002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6672cc5-2340-4262-aeaa-00384ba0e11e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:14.333683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6672cc5-2340-4262-aeaa-00384ba0e11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:14.379565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d6672cc5-2340-4262-aeaa-00384ba0e11e None None] MFA Rules not processed for user `0ff22fcd20c54ea1924f4a0588b2a94b`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:14.381782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d6672cc5-2340-4262-aeaa-00384ba0e11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:14.421853 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:14.485740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1164/2339] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:14.498612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aade79-5c28-4e2f-8bb5-1a2456810ee5 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:14.500396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-82aade79-5c28-4e2f-8bb5-1a2456810ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:14.661367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aade79-5c28-4e2f-8bb5-1a2456810ee5 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:14.667693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca668e-0480-4de1-9b1e-d1f310487f38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:14.668735 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca668e-0480-4de1-9b1e-d1f310487f38 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:14.669311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca668e-0480-4de1-9b1e-d1f310487f38 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:14.669931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca668e-0480-4de1-9b1e-d1f310487f38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:14.777679 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca668e-0480-4de1-9b1e-d1f310487f38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:14.779144 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca668e-0480-4de1-9b1e-d1f310487f38 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:14.802789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a1d52-9f09-42e6-b747-9b89ca0dc6ce None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:14.803273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a1d52-9f09-42e6-b747-9b89ca0dc6ce None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:14.803365 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a1d52-9f09-42e6-b747-9b89ca0dc6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:14.809194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1176/2340] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:14.846855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-256a1d52-9f09-42e6-b747-9b89ca0dc6ce None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:14.848818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-256a1d52-9f09-42e6-b747-9b89ca0dc6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:14.926095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1165/2341] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 1 19:55:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:55:14.941380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca668e-0480-4de1-9b1e-d1f310487f38 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:14.949744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ebca668e-0480-4de1-9b1e-d1f310487f38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:15.042931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca668e-0480-4de1-9b1e-d1f310487f38 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:15.049992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277cd3a-724c-4e0d-b30e-13e30825a780 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:15.051762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277cd3a-724c-4e0d-b30e-13e30825a780 service swift] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:15.052606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277cd3a-724c-4e0d-b30e-13e30825a780 service swift] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:15.053033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277cd3a-724c-4e0d-b30e-13e30825a780 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:15.062198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:15.067450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277cd3a-724c-4e0d-b30e-13e30825a780 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:15.070053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277cd3a-724c-4e0d-b30e-13e30825a780 service swift] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:15.079546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1177/2342] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:55:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:55:16.522030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a1d52-9f09-42e6-b747-9b89ca0dc6ce None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:16.523620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-256a1d52-9f09-42e6-b747-9b89ca0dc6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:16.543380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a1d52-9f09-42e6-b747-9b89ca0dc6ce None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:16.548111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784d6ff-710c-4285-8fdc-3c0559a37400 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:16.549212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784d6ff-710c-4285-8fdc-3c0559a37400 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:16.549721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784d6ff-710c-4285-8fdc-3c0559a37400 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:16.550223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784d6ff-710c-4285-8fdc-3c0559a37400 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:16.644829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784d6ff-710c-4285-8fdc-3c0559a37400 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:16.646895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784d6ff-710c-4285-8fdc-3c0559a37400 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:16.657600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1166/2343] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:17.414810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277cd3a-724c-4e0d-b30e-13e30825a780 service swift] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:17.432106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277cd3a-724c-4e0d-b30e-13e30825a780 service swift] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:17.437149 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb0f5e-206e-4df1-8a36-e18284be5c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:17.438101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb0f5e-206e-4df1-8a36-e18284be5c80 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:17.438530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb0f5e-206e-4df1-8a36-e18284be5c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:17.438953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb0f5e-206e-4df1-8a36-e18284be5c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:17.456651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb0f5e-206e-4df1-8a36-e18284be5c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:17.458505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb0f5e-206e-4df1-8a36-e18284be5c80 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:17.481964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1178/2344] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:18.338311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb6a1e-b236-49b9-92aa-02f43b9fdf2e None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:18.339633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb6a1e-b236-49b9-92aa-02f43b9fdf2e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:18.339922 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb6a1e-b236-49b9-92aa-02f43b9fdf2e None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:18.341987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1167/2345] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:55:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:55:18.356422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc920d74-d7ea-4b41-b394-0ad287176e54 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:18.357100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc920d74-d7ea-4b41-b394-0ad287176e54 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:18.357449 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc920d74-d7ea-4b41-b394-0ad287176e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:18.385619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-cc920d74-d7ea-4b41-b394-0ad287176e54 None None] MFA Rules not processed for user `bba7d9205e694f2182d6091061d1219c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:18.388220 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-cc920d74-d7ea-4b41-b394-0ad287176e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:18.453738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb6a1e-b236-49b9-92aa-02f43b9fdf2e None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:18.455660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-41eb6a1e-b236-49b9-92aa-02f43b9fdf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:18.492338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1179/2346] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:55:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:18.508276 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc920d74-d7ea-4b41-b394-0ad287176e54 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:18.510112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-cc920d74-d7ea-4b41-b394-0ad287176e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:18.524112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc920d74-d7ea-4b41-b394-0ad287176e54 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:18.530059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace42268-8395-42ad-abf6-b88dde55f503 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:18.530892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace42268-8395-42ad-abf6-b88dde55f503 service placement] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:18.531175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace42268-8395-42ad-abf6-b88dde55f503 service placement] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:18.531445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace42268-8395-42ad-abf6-b88dde55f503 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:18.543302 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace42268-8395-42ad-abf6-b88dde55f503 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:18.545365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace42268-8395-42ad-abf6-b88dde55f503 service placement] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:18.558567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1180/2347] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:18.637822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb6a1e-b236-49b9-92aa-02f43b9fdf2e None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:18.644267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:18.645244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:18.645526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:18.645833 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] PATH_INFO: `/v3/users/e8bbb1244dca43f8947835d01b69da0b` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:18.647088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8bbb1244dca43f8947835d01b69da0b)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:18.649049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:18.654052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:18.737480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8bbb1244dca43f8947835d01b69da0b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:18.753776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8bbb1244dca43f8947835d01b69da0b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:18.764532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8bbb1244dca43f8947835d01b69da0b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:18.774844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8bbb1244dca43f8947835d01b69da0b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:18.776195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-bf101ade-6f90-41bb-9863-41a45593f4aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8bbb1244dca43f8947835d01b69da0b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:18.782857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1168/2348] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:55:18 2022] DELETE /identity/v3/users/e8bbb1244dca43f8947835d01b69da0b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:55:18.996461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace42268-8395-42ad-abf6-b88dde55f503 service placement] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:19.015168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace42268-8395-42ad-abf6-b88dde55f503 service placement] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:19.096269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:19.097417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.097816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.098099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] PATH_INFO: `/v3/projects/387f55f94c784e988f8c0e240f2dc1c6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.099357 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387f55f94c784e988f8c0e240f2dc1c6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:19.101291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:19.161410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387f55f94c784e988f8c0e240f2dc1c6', 'request_id': 'req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:19.170728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387f55f94c784e988f8c0e240f2dc1c6', 'request_id': 'req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:19.177983 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387f55f94c784e988f8c0e240f2dc1c6', 'request_id': 'req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:19.181961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:19.193477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] The token cache is being invalidate because project 387f55f94c784e988f8c0e240f2dc1c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:55:19.194199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:19.195473 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a72eae83-5af7-46b6-ba9a-8df582e2c2f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:19.197337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1181/2349] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:55:18 2022] DELETE /identity/v3/projects/387f55f94c784e988f8c0e240f2dc1c6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:19.202295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a17cc5-b583-48ac-bf79-8cdd0160ae0b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.202654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a17cc5-b583-48ac-bf79-8cdd0160ae0b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.202998 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a17cc5-b583-48ac-bf79-8cdd0160ae0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.221505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-34a17cc5-b583-48ac-bf79-8cdd0160ae0b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:19.222916 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-34a17cc5-b583-48ac-bf79-8cdd0160ae0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:19.289857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1169/2350] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:19.297450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ee3c3-96ee-476a-8b8a-c8048856b301 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.297780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ee3c3-96ee-476a-8b8a-c8048856b301 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.298060 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ee3c3-96ee-476a-8b8a-c8048856b301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.318901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-797ee3c3-96ee-476a-8b8a-c8048856b301 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:19.320137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-797ee3c3-96ee-476a-8b8a-c8048856b301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:19.329464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1182/2351] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:19.334746 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a17cc5-b583-48ac-bf79-8cdd0160ae0b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:19.344306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a17cc5-b583-48ac-bf79-8cdd0160ae0b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:19.349851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00534f-fb02-44e9-83b1-ccb6397cc577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:19.350435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00534f-fb02-44e9-83b1-ccb6397cc577 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.350781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00534f-fb02-44e9-83b1-ccb6397cc577 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.350987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00534f-fb02-44e9-83b1-ccb6397cc577 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.351698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00534f-fb02-44e9-83b1-ccb6397cc577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:55:19.351996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00534f-fb02-44e9-83b1-ccb6397cc577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:19.353600 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00534f-fb02-44e9-83b1-ccb6397cc577 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:19.365562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1170/2352] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:55:19.370141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ee3c3-96ee-476a-8b8a-c8048856b301 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:19.385337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ee3c3-96ee-476a-8b8a-c8048856b301 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:19.390785 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafd7ac-bddf-44a0-b211-3f2a7e631d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:19.392056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafd7ac-bddf-44a0-b211-3f2a7e631d70 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.392344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafd7ac-bddf-44a0-b211-3f2a7e631d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.392619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafd7ac-bddf-44a0-b211-3f2a7e631d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.393308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafd7ac-bddf-44a0-b211-3f2a7e631d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:19.395792 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafd7ac-bddf-44a0-b211-3f2a7e631d70 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:19.424376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1183/2353] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:55:19 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:55:19.428876 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00534f-fb02-44e9-83b1-ccb6397cc577 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:19.439672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00534f-fb02-44e9-83b1-ccb6397cc577 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:19.501218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80fd88-afd3-4954-bbfc-ddf52f6acf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:19.501930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80fd88-afd3-4954-bbfc-ddf52f6acf7c admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.502213 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80fd88-afd3-4954-bbfc-ddf52f6acf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.502488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80fd88-afd3-4954-bbfc-ddf52f6acf7c admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.503298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80fd88-afd3-4954-bbfc-ddf52f6acf7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:19.505881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80fd88-afd3-4954-bbfc-ddf52f6acf7c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:19.518018 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:19.531500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1171/2354] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:55:19 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:55:19.536069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafd7ac-bddf-44a0-b211-3f2a7e631d70 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:19.546328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafd7ac-bddf-44a0-b211-3f2a7e631d70 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:19.551716 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2b0fe-fca2-4944-9490-707136ba8364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:19.552444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2b0fe-fca2-4944-9490-707136ba8364 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.552823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2b0fe-fca2-4944-9490-707136ba8364 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.553159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2b0fe-fca2-4944-9490-707136ba8364 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.553958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2b0fe-fca2-4944-9490-707136ba8364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:19.555595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2b0fe-fca2-4944-9490-707136ba8364 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:19.567393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1184/2355] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:19.571443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80fd88-afd3-4954-bbfc-ddf52f6acf7c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:19.582528 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80fd88-afd3-4954-bbfc-ddf52f6acf7c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:19.587137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba934945-c9d6-4066-a8e9-5979ec250eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:19.587785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba934945-c9d6-4066-a8e9-5979ec250eac admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.588060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba934945-c9d6-4066-a8e9-5979ec250eac admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.588330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba934945-c9d6-4066-a8e9-5979ec250eac admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.589050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba934945-c9d6-4066-a8e9-5979ec250eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:19.590683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba934945-c9d6-4066-a8e9-5979ec250eac admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:19.602000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1172/2356] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:19.605197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2b0fe-fca2-4944-9490-707136ba8364 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:19.615991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2b0fe-fca2-4944-9490-707136ba8364 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:19.620948 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa460c-1cda-45a7-9517-7b222add8328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:19.621889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa460c-1cda-45a7-9517-7b222add8328 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.622164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa460c-1cda-45a7-9517-7b222add8328 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.622433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa460c-1cda-45a7-9517-7b222add8328 admin admin] PATH_INFO: `/v3/projects/0e1ce779cd384cceb475c96e2f2442c0/users/7cee7c401ecf4673934428c4925d7060/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.638616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa460c-1cda-45a7-9517-7b222add8328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e1ce779cd384cceb475c96e2f2442c0, user_id=7cee7c401ecf4673934428c4925d7060, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:19.641423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa460c-1cda-45a7-9517-7b222add8328 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:19.658002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1185/2357] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:55:19 2022] PUT /identity/v3/projects/0e1ce779cd384cceb475c96e2f2442c0/users/7cee7c401ecf4673934428c4925d7060/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:19.673163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d171c7-a51a-4f69-ad5b-8418785d8c8a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.673449 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d171c7-a51a-4f69-ad5b-8418785d8c8a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.673757 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d171c7-a51a-4f69-ad5b-8418785d8c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.726784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-59d171c7-a51a-4f69-ad5b-8418785d8c8a None None] MFA Rules not processed for user `7cee7c401ecf4673934428c4925d7060`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:19.728371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-59d171c7-a51a-4f69-ad5b-8418785d8c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:19.806798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1173/2358] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:19 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:19.822042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa460c-1cda-45a7-9517-7b222add8328 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:19.823889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7efa460c-1cda-45a7-9517-7b222add8328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:19.899813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa460c-1cda-45a7-9517-7b222add8328 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:19.903907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f755b6b-88df-4b61-9930-2a83f3ca57ba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c022224b03a54ab489d95502c63a25dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:19.904772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f755b6b-88df-4b61-9930-2a83f3ca57ba service glance] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:19.905132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f755b6b-88df-4b61-9930-2a83f3ca57ba service glance] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:19.905483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f755b6b-88df-4b61-9930-2a83f3ca57ba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:19.915702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f755b6b-88df-4b61-9930-2a83f3ca57ba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:19.917167 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f755b6b-88df-4b61-9930-2a83f3ca57ba service glance] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:19.928399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1186/2359] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:19 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:21.287417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80e9b-c193-428b-9e16-338db37e9617 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:21.288310 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80e9b-c193-428b-9e16-338db37e9617 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:21.288754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c80e9b-c193-428b-9e16-338db37e9617 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:21.290179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1174/2360] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:55:21.296585 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d850e0-d5a9-479f-a307-8160ef9284cf None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:21.297117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d850e0-d5a9-479f-a307-8160ef9284cf None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:21.297117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d850e0-d5a9-479f-a307-8160ef9284cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:21.324464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d2d850e0-d5a9-479f-a307-8160ef9284cf None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:21.326941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d2d850e0-d5a9-479f-a307-8160ef9284cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:21.415198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1187/2361] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:55:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:21.689867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c80e9b-c193-428b-9e16-338db37e9617 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:21.692844 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-79c80e9b-c193-428b-9e16-338db37e9617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:21.766539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c80e9b-c193-428b-9e16-338db37e9617 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:21.771898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:21.772785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:21.773065 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:21.773340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] PATH_INFO: `/v3/users/5086331995784770b4615804398691c6` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:21.774581 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5086331995784770b4615804398691c6)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:21.776818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:21.829901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5086331995784770b4615804398691c6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:21.842815 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5086331995784770b4615804398691c6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:21.850949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5086331995784770b4615804398691c6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:21.859912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:21.859912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5086331995784770b4615804398691c6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:21.860215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5086331995784770b4615804398691c6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:21.868842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1175/2362] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:55:21 2022] DELETE /identity/v3/users/5086331995784770b4615804398691c6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:21.873566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f08368-7bce-4a14-8fdf-1f583f7f1454 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:21.873911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f08368-7bce-4a14-8fdf-1f583f7f1454 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:21.874190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f08368-7bce-4a14-8fdf-1f583f7f1454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:21.895676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-93f08368-7bce-4a14-8fdf-1f583f7f1454 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:21.897002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-93f08368-7bce-4a14-8fdf-1f583f7f1454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:21.916299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:21.961350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1188/2363] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:21.973008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:21.974490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:22.055003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68c8f5-bb72-445a-b8a9-5a173a09b288 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:22.058648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65d69e-4d24-4a49-b0a1-9722a0f9df84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:22.059283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65d69e-4d24-4a49-b0a1-9722a0f9df84 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:22.059563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65d69e-4d24-4a49-b0a1-9722a0f9df84 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:22.059865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65d69e-4d24-4a49-b0a1-9722a0f9df84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:22.069842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65d69e-4d24-4a49-b0a1-9722a0f9df84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:22.071269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65d69e-4d24-4a49-b0a1-9722a0f9df84 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:22.081100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1176/2364] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:22.441242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f08368-7bce-4a14-8fdf-1f583f7f1454 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:22.449806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-93f08368-7bce-4a14-8fdf-1f583f7f1454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:22.477620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f08368-7bce-4a14-8fdf-1f583f7f1454 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:22.482684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:22.483570 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:22.483955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:22.484113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] PATH_INFO: `/v3/projects/13acfdc03c10433aac900052a1355e78` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:22.486042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13acfdc03c10433aac900052a1355e78)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:22.488784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:22.547656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13acfdc03c10433aac900052a1355e78', 'request_id': 'req-0c78a1c5-c897-4925-90dc-850e52b94133'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:22.556693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13acfdc03c10433aac900052a1355e78', 'request_id': 'req-0c78a1c5-c897-4925-90dc-850e52b94133'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:22.563483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13acfdc03c10433aac900052a1355e78', 'request_id': 'req-0c78a1c5-c897-4925-90dc-850e52b94133'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:22.576331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] The token cache is being invalidate because project 13acfdc03c10433aac900052a1355e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:55:22.576626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:22.577280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:22.579343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1189/2365] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:55:22 2022] DELETE /identity/v3/projects/13acfdc03c10433aac900052a1355e78 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:22.583752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65d69e-4d24-4a49-b0a1-9722a0f9df84 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:22.594401 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65d69e-4d24-4a49-b0a1-9722a0f9df84 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:22.655153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:22.656118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:22.656303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:22.656573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] PATH_INFO: `/v3/users/e7f342dc594d4bdbb5a75500829b8074` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:22.658029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f342dc594d4bdbb5a75500829b8074)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:22.660682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:22.710688 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f342dc594d4bdbb5a75500829b8074'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:22.723813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f342dc594d4bdbb5a75500829b8074'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:22.731171 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f342dc594d4bdbb5a75500829b8074'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:22.736687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:22.741113 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f342dc594d4bdbb5a75500829b8074'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:22.742292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f342dc594d4bdbb5a75500829b8074'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:22.749652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1177/2366] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:55:22 2022] DELETE /identity/v3/users/e7f342dc594d4bdbb5a75500829b8074 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:23.054197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:23.055782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:23.069508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78a1c5-c897-4925-90dc-850e52b94133 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:23.131061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:23.131938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:23.132220 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:23.132491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] PATH_INFO: `/v3/projects/730d7caf0db24ceaa194788743542408` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:23.133744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=730d7caf0db24ceaa194788743542408)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:23.135661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:23.184922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '730d7caf0db24ceaa194788743542408', 'request_id': 'req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.187106 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:23.193976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '730d7caf0db24ceaa194788743542408', 'request_id': 'req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.201493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '730d7caf0db24ceaa194788743542408', 'request_id': 'req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.212899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] The token cache is being invalidate because project 730d7caf0db24ceaa194788743542408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:55:23.213286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.214027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.215722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1190/2367] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:55:23 2022] DELETE /identity/v3/projects/730d7caf0db24ceaa194788743542408 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:23.219693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:23.221085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:23.234797 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fa654-62a2-49e8-9125-495a9ea0d20f admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:23.294622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:23.295507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:23.295783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:23.296052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] PATH_INFO: `/v3/users/0ff22fcd20c54ea1924f4a0588b2a94b` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:23.297449 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ff22fcd20c54ea1924f4a0588b2a94b)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:23.299406 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:23.346725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ff22fcd20c54ea1924f4a0588b2a94b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.358742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ff22fcd20c54ea1924f4a0588b2a94b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.365857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ff22fcd20c54ea1924f4a0588b2a94b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.372747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ff22fcd20c54ea1924f4a0588b2a94b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.374185 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-122580ae-340d-4aab-8d80-c3f7b116a48c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ff22fcd20c54ea1924f4a0588b2a94b'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.381372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1178/2368] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:55:23 2022] DELETE /identity/v3/users/0ff22fcd20c54ea1924f4a0588b2a94b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:23.817072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:23.818700 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:23.832754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c03d93-ddeb-45d8-b1d4-d2d28dec3fc6 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:23.904618 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:23.905658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:23.906044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:23.906430 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] PATH_INFO: `/v3/projects/20d02d6730944ed4b81cc57c4ed051fc` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:23.907744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20d02d6730944ed4b81cc57c4ed051fc)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:23.909936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:23.958739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20d02d6730944ed4b81cc57c4ed051fc', 'request_id': 'req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.969629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20d02d6730944ed4b81cc57c4ed051fc', 'request_id': 'req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.976518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20d02d6730944ed4b81cc57c4ed051fc', 'request_id': 'req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.986696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] The token cache is being invalidate because project 20d02d6730944ed4b81cc57c4ed051fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:55:23.986982 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.988259 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d8c702a6-dd57-4d3a-b972-eb1d570b48e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:55:23.989861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1191/2369] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:55:23 2022] DELETE /identity/v3/projects/20d02d6730944ed4b81cc57c4ed051fc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:23.994997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be138c9d-47a1-4d5f-b9db-1bdcccd223d6 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:23.995467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be138c9d-47a1-4d5f-b9db-1bdcccd223d6 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:23.995467 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be138c9d-47a1-4d5f-b9db-1bdcccd223d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:24.027692 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-be138c9d-47a1-4d5f-b9db-1bdcccd223d6 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:24.029044 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-be138c9d-47a1-4d5f-b9db-1bdcccd223d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:24.090912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1179/2370] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:24.099548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322c152-92f2-4248-8cb0-8f8b53b70934 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:24.099839 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322c152-92f2-4248-8cb0-8f8b53b70934 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:24.100113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322c152-92f2-4248-8cb0-8f8b53b70934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:24.120519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-8322c152-92f2-4248-8cb0-8f8b53b70934 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:24.122260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8322c152-92f2-4248-8cb0-8f8b53b70934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:24.131168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1192/2371] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:24.135532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be138c9d-47a1-4d5f-b9db-1bdcccd223d6 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:24.146120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be138c9d-47a1-4d5f-b9db-1bdcccd223d6 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:24.151156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2982f2-37fe-4bc9-897d-9825323dbca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:24.151889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2982f2-37fe-4bc9-897d-9825323dbca1 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:24.152349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2982f2-37fe-4bc9-897d-9825323dbca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:24.152584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2982f2-37fe-4bc9-897d-9825323dbca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:24.153380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2982f2-37fe-4bc9-897d-9825323dbca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:55:24.153870 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2982f2-37fe-4bc9-897d-9825323dbca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:24.155579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2982f2-37fe-4bc9-897d-9825323dbca1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:24.157387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:24.171582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1180/2372] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:55:24.176412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322c152-92f2-4248-8cb0-8f8b53b70934 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:24.185875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322c152-92f2-4248-8cb0-8f8b53b70934 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:24.190744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e375e68-4d5b-4e02-8d54-5d8cb1584639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:24.191399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e375e68-4d5b-4e02-8d54-5d8cb1584639 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:24.191681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e375e68-4d5b-4e02-8d54-5d8cb1584639 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:24.191957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e375e68-4d5b-4e02-8d54-5d8cb1584639 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:24.192658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e375e68-4d5b-4e02-8d54-5d8cb1584639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:24.194615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e375e68-4d5b-4e02-8d54-5d8cb1584639 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:24.231756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1193/2373] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:55:24 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:55:24.236183 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2982f2-37fe-4bc9-897d-9825323dbca1 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:24.247272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2982f2-37fe-4bc9-897d-9825323dbca1 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:24.317192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94d470-75df-4fce-9344-3654ac5d87b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:24.318287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94d470-75df-4fce-9344-3654ac5d87b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:24.318287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94d470-75df-4fce-9344-3654ac5d87b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:24.318521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94d470-75df-4fce-9344-3654ac5d87b7 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:24.319422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94d470-75df-4fce-9344-3654ac5d87b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:24.321939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94d470-75df-4fce-9344-3654ac5d87b7 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:24.343898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1181/2374] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:55:24 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:55:24.348626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e375e68-4d5b-4e02-8d54-5d8cb1584639 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:24.358853 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e375e68-4d5b-4e02-8d54-5d8cb1584639 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:24.365185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d172ad5-3c56-4186-8765-16e7f07e17c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:24.366015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172ad5-3c56-4186-8765-16e7f07e17c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:24.366296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172ad5-3c56-4186-8765-16e7f07e17c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:24.366565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d172ad5-3c56-4186-8765-16e7f07e17c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:24.367349 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d172ad5-3c56-4186-8765-16e7f07e17c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:24.368990 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d172ad5-3c56-4186-8765-16e7f07e17c7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:24.380688 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1194/2375] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:24.384965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94d470-75df-4fce-9344-3654ac5d87b7 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:24.396795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94d470-75df-4fce-9344-3654ac5d87b7 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:24.401944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e99fd-b712-4abb-974c-7b7dc851ce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:24.402611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e99fd-b712-4abb-974c-7b7dc851ce7c admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:24.402895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e99fd-b712-4abb-974c-7b7dc851ce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:24.403172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e99fd-b712-4abb-974c-7b7dc851ce7c admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:24.403901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e99fd-b712-4abb-974c-7b7dc851ce7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:24.405510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e99fd-b712-4abb-974c-7b7dc851ce7c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:24.416609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1182/2376] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:55:24.425488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d172ad5-3c56-4186-8765-16e7f07e17c7 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:24.435749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d172ad5-3c56-4186-8765-16e7f07e17c7 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:24.441143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2570b-fb0a-43ba-9fac-936e88910497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:24.442158 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2570b-fb0a-43ba-9fac-936e88910497 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:24.442471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2570b-fb0a-43ba-9fac-936e88910497 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:24.442756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2570b-fb0a-43ba-9fac-936e88910497 admin admin] PATH_INFO: `/v3/projects/7e0ee432504e4858b4287cbd2e318549/users/e1dfe324dea649d49ac6ee4f9d2eec20/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:24.458793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2570b-fb0a-43ba-9fac-936e88910497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0ee432504e4858b4287cbd2e318549, user_id=e1dfe324dea649d49ac6ee4f9d2eec20, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:24.461457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2570b-fb0a-43ba-9fac-936e88910497 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:24.476465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1195/2377] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:55:24 2022] PUT /identity/v3/projects/7e0ee432504e4858b4287cbd2e318549/users/e1dfe324dea649d49ac6ee4f9d2eec20/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:55:24.546510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ad5b-9795-4ec9-94b3-9159b05d5990 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:24.546762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ad5b-9795-4ec9-94b3-9159b05d5990 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:24.547043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ad5b-9795-4ec9-94b3-9159b05d5990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:24.579817 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-3f68ad5b-9795-4ec9-94b3-9159b05d5990 None None] MFA Rules not processed for user `e1dfe324dea649d49ac6ee4f9d2eec20`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:24.581026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3f68ad5b-9795-4ec9-94b3-9159b05d5990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:24.666892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1183/2378] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:24.679523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2570b-fb0a-43ba-9fac-936e88910497 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:24.681278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2dc2570b-fb0a-43ba-9fac-936e88910497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:24.687810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:24.751504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2570b-fb0a-43ba-9fac-936e88910497 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:24.758413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7d65c-afc1-4da5-8d1a-7775f55ac77f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:24.759257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d65c-afc1-4da5-8d1a-7775f55ac77f service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:24.759635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d65c-afc1-4da5-8d1a-7775f55ac77f service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:24.760159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d65c-afc1-4da5-8d1a-7775f55ac77f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:24.774638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7d65c-afc1-4da5-8d1a-7775f55ac77f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:24.776609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7d65c-afc1-4da5-8d1a-7775f55ac77f service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:24.792518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1196/2379] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:28.860810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68ad5b-9795-4ec9-94b3-9159b05d5990 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:28.862400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3f68ad5b-9795-4ec9-94b3-9159b05d5990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:28.923290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68ad5b-9795-4ec9-94b3-9159b05d5990 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:28.928348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bfcc9-e355-48d5-bcdf-46c0ff7246b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c022224b03a54ab489d95502c63a25dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:28.929009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bfcc9-e355-48d5-bcdf-46c0ff7246b2 service glance] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:28.929285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bfcc9-e355-48d5-bcdf-46c0ff7246b2 service glance] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:28.929554 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bfcc9-e355-48d5-bcdf-46c0ff7246b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:28.931374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-363bfcc9-e355-48d5-bcdf-46c0ff7246b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:28.947323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:29.007335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bfcc9-e355-48d5-bcdf-46c0ff7246b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:29.008821 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bfcc9-e355-48d5-bcdf-46c0ff7246b2 service glance] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:29.020122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1184/2380] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:29.100945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61833b5a-b36d-4f83-b8c7-b3120e7eac6a None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:29.103206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61833b5a-b36d-4f83-b8c7-b3120e7eac6a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:29.103690 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61833b5a-b36d-4f83-b8c7-b3120e7eac6a None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:29.105633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1197/2381] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:55:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:55:29.111150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace12d4-214a-4b6e-bdff-07cf0d6fbb9a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:29.111438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace12d4-214a-4b6e-bdff-07cf0d6fbb9a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:29.111719 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace12d4-214a-4b6e-bdff-07cf0d6fbb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:29.146772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-6ace12d4-214a-4b6e-bdff-07cf0d6fbb9a None None] MFA Rules not processed for user `370f1c0f7af04a56bdf8fa452550380a`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:29.147986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6ace12d4-214a-4b6e-bdff-07cf0d6fbb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:29.200203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1185/2382] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:55:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:55:29.213159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61833b5a-b36d-4f83-b8c7-b3120e7eac6a None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:29.214943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-61833b5a-b36d-4f83-b8c7-b3120e7eac6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:29.228833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61833b5a-b36d-4f83-b8c7-b3120e7eac6a None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:29.232411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7173e86-9fe9-4047-a0f2-1e21b550b6dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:29.233051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7173e86-9fe9-4047-a0f2-1e21b550b6dc service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:29.233330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7173e86-9fe9-4047-a0f2-1e21b550b6dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:29.233631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7173e86-9fe9-4047-a0f2-1e21b550b6dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:29.241255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7173e86-9fe9-4047-a0f2-1e21b550b6dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:29.242686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7173e86-9fe9-4047-a0f2-1e21b550b6dc service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:29.250656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1198/2383] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:29.336063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bfcd6-2cb1-43d0-b671-650d488e6f8a None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:29.336645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bfcd6-2cb1-43d0-b671-650d488e6f8a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:29.336947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bfcd6-2cb1-43d0-b671-650d488e6f8a None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:29.338361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1186/2384] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 19:55:29.344299 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf0826-7a78-4da0-a23e-7f2a802161a5 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:29.344582 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf0826-7a78-4da0-a23e-7f2a802161a5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:29.344855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf0826-7a78-4da0-a23e-7f2a802161a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:29.365071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-e9bf0826-7a78-4da0-a23e-7f2a802161a5 None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:29.366306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e9bf0826-7a78-4da0-a23e-7f2a802161a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:29.375735 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1199/2385] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:55:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:29.392341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bfcd6-2cb1-43d0-b671-650d488e6f8a None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:29.394576 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d97bfcd6-2cb1-43d0-b671-650d488e6f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:29.409669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bfcd6-2cb1-43d0-b671-650d488e6f8a None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:29.414403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6315c-8825-423c-b05a-9485e984b2cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:29.415035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6315c-8825-423c-b05a-9485e984b2cf service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:29.415312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6315c-8825-423c-b05a-9485e984b2cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:29.415596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6315c-8825-423c-b05a-9485e984b2cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:29.427410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6315c-8825-423c-b05a-9485e984b2cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:29.428979 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6315c-8825-423c-b05a-9485e984b2cf service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:29.440732 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1187/2386] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:55:35.898917 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3853d2-8628-4776-8764-639c39c46c55 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:35.899476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3853d2-8628-4776-8764-639c39c46c55 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:35.899476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3853d2-8628-4776-8764-639c39c46c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:35.919336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7b3853d2-8628-4776-8764-639c39c46c55 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:35.920553 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7b3853d2-8628-4776-8764-639c39c46c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:35.982529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1200/2387] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:55:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:35.995027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6315c-8825-423c-b05a-9485e984b2cf service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:35.996419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-59d6315c-8825-423c-b05a-9485e984b2cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:36.064727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6315c-8825-423c-b05a-9485e984b2cf service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:36.070863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74655278-5ca3-4769-ae84-6b817f79c14f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:36.071688 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74655278-5ca3-4769-ae84-6b817f79c14f service swift] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:36.072070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74655278-5ca3-4769-ae84-6b817f79c14f service swift] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:36.072356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74655278-5ca3-4769-ae84-6b817f79c14f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:36.080933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74655278-5ca3-4769-ae84-6b817f79c14f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:36.083260 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74655278-5ca3-4769-ae84-6b817f79c14f service swift] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:36.092183 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1188/2388] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:55:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:55:37.870068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b15e8-6067-476b-a355-9383e944a05a None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:37.871008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b15e8-6067-476b-a355-9383e944a05a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:37.871008 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b15e8-6067-476b-a355-9383e944a05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:37.890033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-104b15e8-6067-476b-a355-9383e944a05a None None] MFA Rules not processed for user `80267c31f3fc4b3fbc8dc7ed03326eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:55:37.891463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-104b15e8-6067-476b-a355-9383e944a05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:37.960881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1201/2389] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:55:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:55:46.088851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=91 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:47.953382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=86 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:55:48.998639 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74655278-5ca3-4769-ae84-6b817f79c14f service swift] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:55:49.000071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-74655278-5ca3-4769-ae84-6b817f79c14f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:49.119522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74655278-5ca3-4769-ae84-6b817f79c14f service swift] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:55:49.125239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7c6c0-c64e-4d08-a93e-f26b4763e46b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:55:49.125986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c6c0-c64e-4d08-a93e-f26b4763e46b service placement] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:55:49.126244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c6c0-c64e-4d08-a93e-f26b4763e46b service placement] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:55:49.126575 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7c6c0-c64e-4d08-a93e-f26b4763e46b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:55:49.129454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e2e7c6c0-c64e-4d08-a93e-f26b4763e46b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:55:49.141120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7c6c0-c64e-4d08-a93e-f26b4763e46b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:55:49.142904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7c6c0-c64e-4d08-a93e-f26b4763e46b service placement] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:55:49.154568 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1189/2390] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:55:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:55:59.149903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=41 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:03.242233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b15e8-6067-476b-a355-9383e944a05a None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:03.243776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-104b15e8-6067-476b-a355-9383e944a05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:03.346403 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b15e8-6067-476b-a355-9383e944a05a None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:03.352628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:03.353694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:03.354047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:03.354500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] PATH_INFO: `/v3/users/7cee7c401ecf4673934428c4925d7060` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:03.355965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cee7c401ecf4673934428c4925d7060)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:03.358287 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:03.435227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cee7c401ecf4673934428c4925d7060'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:03.445246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cee7c401ecf4673934428c4925d7060'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:03.446228 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cee7c401ecf4673934428c4925d7060'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:03.460759 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cee7c401ecf4673934428c4925d7060'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:03.461440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cee7c401ecf4673934428c4925d7060'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:03.475342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1202/2391] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:56:03 2022] DELETE /identity/v3/users/7cee7c401ecf4673934428c4925d7060 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:56:03.479875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db207f4-80d0-416d-be1a-f085aaaf5f86 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:03.480383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db207f4-80d0-416d-be1a-f085aaaf5f86 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:03.480492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db207f4-80d0-416d-be1a-f085aaaf5f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:03.499100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-4db207f4-80d0-416d-be1a-f085aaaf5f86 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:03.500320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4db207f4-80d0-416d-be1a-f085aaaf5f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:03.575515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1190/2392] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:03.585483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:03.586973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:03.648883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79bc2d-d6a3-43e0-9cac-24e7d104363e admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:03.652662 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fef0c2-4242-4595-8d6f-4d7eb79199e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:03.653331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fef0c2-4242-4595-8d6f-4d7eb79199e7 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:03.653639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fef0c2-4242-4595-8d6f-4d7eb79199e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:03.653921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fef0c2-4242-4595-8d6f-4d7eb79199e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:03.663752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fef0c2-4242-4595-8d6f-4d7eb79199e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:03.665183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fef0c2-4242-4595-8d6f-4d7eb79199e7 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:03.675040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1203/2393] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:56:03.968992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db207f4-80d0-416d-be1a-f085aaaf5f86 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:03.970448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4db207f4-80d0-416d-be1a-f085aaaf5f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:03.983163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db207f4-80d0-416d-be1a-f085aaaf5f86 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:03.988351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:03.989787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:03.990072 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:03.990355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] PATH_INFO: `/v3/projects/0e1ce779cd384cceb475c96e2f2442c0` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:03.991565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e1ce779cd384cceb475c96e2f2442c0)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:03.993473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:04.041541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e1ce779cd384cceb475c96e2f2442c0', 'request_id': 'req-ad0170f0-46fe-4692-b66a-8cfb478b33b6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:04.048978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e1ce779cd384cceb475c96e2f2442c0', 'request_id': 'req-ad0170f0-46fe-4692-b66a-8cfb478b33b6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:04.055320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e1ce779cd384cceb475c96e2f2442c0', 'request_id': 'req-ad0170f0-46fe-4692-b66a-8cfb478b33b6'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:04.066344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] The token cache is being invalidate because project 0e1ce779cd384cceb475c96e2f2442c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:56:04.066573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:04.067839 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ad0170f0-46fe-4692-b66a-8cfb478b33b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:04.069721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1191/2394] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:56:03 2022] DELETE /identity/v3/projects/0e1ce779cd384cceb475c96e2f2442c0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:56:04.074674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e12c00-e58d-46ad-8611-a7f0bf5ac8c9 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:04.074962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e12c00-e58d-46ad-8611-a7f0bf5ac8c9 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:04.075240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e12c00-e58d-46ad-8611-a7f0bf5ac8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:04.093057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-16e12c00-e58d-46ad-8611-a7f0bf5ac8c9 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:04.094294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-16e12c00-e58d-46ad-8611-a7f0bf5ac8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:04.110741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:04.153784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1204/2395] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:04.161728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4fc98-b919-45ba-9988-8d8e07633a4d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:04.162018 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4fc98-b919-45ba-9988-8d8e07633a4d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:04.162295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4fc98-b919-45ba-9988-8d8e07633a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:04.180707 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-1dc4fc98-b919-45ba-9988-8d8e07633a4d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:04.181946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1dc4fc98-b919-45ba-9988-8d8e07633a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:04.189882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1192/2396] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:04.194812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e12c00-e58d-46ad-8611-a7f0bf5ac8c9 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:04.204398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e12c00-e58d-46ad-8611-a7f0bf5ac8c9 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:04.209136 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d978e-8a57-4ee2-a258-1dde70a4162d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:04.209824 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d978e-8a57-4ee2-a258-1dde70a4162d admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:04.210108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d978e-8a57-4ee2-a258-1dde70a4162d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:04.210385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d978e-8a57-4ee2-a258-1dde70a4162d admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:04.211107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d978e-8a57-4ee2-a258-1dde70a4162d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:56:04.211413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d978e-8a57-4ee2-a258-1dde70a4162d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:04.213020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d978e-8a57-4ee2-a258-1dde70a4162d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:04.224616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1205/2397] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:56:04.228998 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4fc98-b919-45ba-9988-8d8e07633a4d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:04.238221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4fc98-b919-45ba-9988-8d8e07633a4d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:04.243516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc5e09-1084-4082-b273-67cd74676415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:04.244150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc5e09-1084-4082-b273-67cd74676415 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:04.244431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc5e09-1084-4082-b273-67cd74676415 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:04.244704 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc5e09-1084-4082-b273-67cd74676415 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:04.245395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc5e09-1084-4082-b273-67cd74676415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:04.247328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc5e09-1084-4082-b273-67cd74676415 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:04.274680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1193/2398] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:56:04 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:56:04.277996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d978e-8a57-4ee2-a258-1dde70a4162d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:04.288214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d978e-8a57-4ee2-a258-1dde70a4162d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:04.345991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0892c-871b-4c63-a235-b01148e6f3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:04.346663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0892c-871b-4c63-a235-b01148e6f3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:04.346944 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0892c-871b-4c63-a235-b01148e6f3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:04.347248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0892c-871b-4c63-a235-b01148e6f3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:04.347914 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0892c-871b-4c63-a235-b01148e6f3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:04.349860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0892c-871b-4c63-a235-b01148e6f3a8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:04.370486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1206/2399] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:56:04 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:56:04.374610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc5e09-1084-4082-b273-67cd74676415 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:04.385513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc5e09-1084-4082-b273-67cd74676415 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:04.390597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f1a4e-7d81-4418-ac90-c305fad65ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:04.391252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f1a4e-7d81-4418-ac90-c305fad65ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:04.391532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f1a4e-7d81-4418-ac90-c305fad65ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:04.391808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f1a4e-7d81-4418-ac90-c305fad65ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:04.392535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f1a4e-7d81-4418-ac90-c305fad65ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:04.394151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f1a4e-7d81-4418-ac90-c305fad65ba2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:04.405101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1194/2400] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:56:04.409013 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0892c-871b-4c63-a235-b01148e6f3a8 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:04.419889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0892c-871b-4c63-a235-b01148e6f3a8 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:04.424627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d9039-9d3a-4547-bc99-023ac69893f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:04.425528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d9039-9d3a-4547-bc99-023ac69893f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:04.425854 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d9039-9d3a-4547-bc99-023ac69893f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:04.426131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d9039-9d3a-4547-bc99-023ac69893f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:04.429141 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d9039-9d3a-4547-bc99-023ac69893f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:04.430922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d9039-9d3a-4547-bc99-023ac69893f5 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:04.445132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1207/2401] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:56:04.446317 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f1a4e-7d81-4418-ac90-c305fad65ba2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:04.458053 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f1a4e-7d81-4418-ac90-c305fad65ba2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:04.465972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ae3a1-e0f7-4443-98a0-0b683f3006a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:04.467003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ae3a1-e0f7-4443-98a0-0b683f3006a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:04.467285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ae3a1-e0f7-4443-98a0-0b683f3006a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:04.467610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ae3a1-e0f7-4443-98a0-0b683f3006a5 admin admin] PATH_INFO: `/v3/projects/e33de0b0c3244c78a86fbfda1c9fbdd6/users/d4ef9ec1904d4e76ae1cd1dc197e4f41/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:04.483555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ae3a1-e0f7-4443-98a0-0b683f3006a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33de0b0c3244c78a86fbfda1c9fbdd6, user_id=d4ef9ec1904d4e76ae1cd1dc197e4f41, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:04.486429 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ae3a1-e0f7-4443-98a0-0b683f3006a5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:04.502057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1195/2402] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:56:04 2022] PUT /identity/v3/projects/e33de0b0c3244c78a86fbfda1c9fbdd6/users/d4ef9ec1904d4e76ae1cd1dc197e4f41/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:56:04.507344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f7be7-f259-45dd-9f9f-00595973a6af None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:04.507881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f7be7-f259-45dd-9f9f-00595973a6af None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:04.508261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f7be7-f259-45dd-9f9f-00595973a6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:04.532097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-758f7be7-f259-45dd-9f9f-00595973a6af None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:04.533294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-758f7be7-f259-45dd-9f9f-00595973a6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:04.594351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1208/2403] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:04.606420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ae3a1-e0f7-4443-98a0-0b683f3006a5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:04.608234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-330ae3a1-e0f7-4443-98a0-0b683f3006a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:04.634028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:04.673920 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ae3a1-e0f7-4443-98a0-0b683f3006a5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:04.678595 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee10f34-4cdb-4b60-b397-a81bb3ad18ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:04.679228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee10f34-4cdb-4b60-b397-a81bb3ad18ec service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:04.679506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee10f34-4cdb-4b60-b397-a81bb3ad18ec service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:04.679777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee10f34-4cdb-4b60-b397-a81bb3ad18ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:04.688552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee10f34-4cdb-4b60-b397-a81bb3ad18ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:04.690002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee10f34-4cdb-4b60-b397-a81bb3ad18ec service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:04.700960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1196/2404] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:56:05.550701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da3bd2-f660-439b-b297-69853a614e38 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:05.551208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da3bd2-f660-439b-b297-69853a614e38 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:05.551334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da3bd2-f660-439b-b297-69853a614e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:05.599641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:05.616555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f4da3bd2-f660-439b-b297-69853a614e38 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:05.618569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f4da3bd2-f660-439b-b297-69853a614e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:05.731788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1209/2405] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 1 19:56:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:05.764477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee10f34-4cdb-4b60-b397-a81bb3ad18ec service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:05.771999 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4ee10f34-4cdb-4b60-b397-a81bb3ad18ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:05.945997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee10f34-4cdb-4b60-b397-a81bb3ad18ec service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:05.952004 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f61b09-8734-4038-8b26-cbef06380220 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:05.953350 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f61b09-8734-4038-8b26-cbef06380220 service swift] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:05.954243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f61b09-8734-4038-8b26-cbef06380220 service swift] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:05.954243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f61b09-8734-4038-8b26-cbef06380220 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:05.967920 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f61b09-8734-4038-8b26-cbef06380220 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:05.970846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f61b09-8734-4038-8b26-cbef06380220 service swift] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:05.987890 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1197/2406] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:56:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:56:08.030725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da3bd2-f660-439b-b297-69853a614e38 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:08.032377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f4da3bd2-f660-439b-b297-69853a614e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:08.050034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da3bd2-f660-439b-b297-69853a614e38 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:08.058585 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:08.059575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:08.059946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:08.060314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] PATH_INFO: `/v3/users/e1dfe324dea649d49ac6ee4f9d2eec20` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:08.061758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1dfe324dea649d49ac6ee4f9d2eec20)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:08.063764 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:08.133782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1dfe324dea649d49ac6ee4f9d2eec20'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:08.147413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1dfe324dea649d49ac6ee4f9d2eec20'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:08.148644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1dfe324dea649d49ac6ee4f9d2eec20'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:08.170686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1dfe324dea649d49ac6ee4f9d2eec20'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:08.171516 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1dfe324dea649d49ac6ee4f9d2eec20'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:08.193300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1210/2407] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:56:08 2022] DELETE /identity/v3/users/e1dfe324dea649d49ac6ee4f9d2eec20 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:56:08.199013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca34e45-1446-4333-81fd-0b6a54916068 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:08.199462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca34e45-1446-4333-81fd-0b6a54916068 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:08.199650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca34e45-1446-4333-81fd-0b6a54916068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:08.232215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-3ca34e45-1446-4333-81fd-0b6a54916068 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:08.235647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3ca34e45-1446-4333-81fd-0b6a54916068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:08.374187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1198/2408] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:08.391225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:08.394910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:08.531430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca34e45-1446-4333-81fd-0b6a54916068 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:08.533395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3ca34e45-1446-4333-81fd-0b6a54916068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:08.541435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078e179-ccee-42d3-90be-e0187061fd08 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:08.546671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fd05c-d3ca-4a01-9356-0ab75cee9dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:08.547343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fd05c-d3ca-4a01-9356-0ab75cee9dbd service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:08.547624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fd05c-d3ca-4a01-9356-0ab75cee9dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:08.547926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fd05c-d3ca-4a01-9356-0ab75cee9dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:08.559262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca34e45-1446-4333-81fd-0b6a54916068 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:08.562568 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fd05c-d3ca-4a01-9356-0ab75cee9dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:08.564381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fd05c-d3ca-4a01-9356-0ab75cee9dbd service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:08.565152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a574b55-769b-4f8c-af6c-2e5c2ee9fd30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:08.566623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a574b55-769b-4f8c-af6c-2e5c2ee9fd30 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:08.567859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a574b55-769b-4f8c-af6c-2e5c2ee9fd30 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:08.568160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a574b55-769b-4f8c-af6c-2e5c2ee9fd30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:08.570835 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6a574b55-769b-4f8c-af6c-2e5c2ee9fd30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:08.583560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1211/2409] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:56:08.611046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:08.660253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a574b55-769b-4f8c-af6c-2e5c2ee9fd30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:08.662530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a574b55-769b-4f8c-af6c-2e5c2ee9fd30 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:08.678004 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1199/2410] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:56:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:56:09.447338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812523fb-932f-494b-b1c7-322623cc03dc None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:09.448425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812523fb-932f-494b-b1c7-322623cc03dc None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:09.448982 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812523fb-932f-494b-b1c7-322623cc03dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:09.486588 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-812523fb-932f-494b-b1c7-322623cc03dc None None] MFA Rules not processed for user `d4ef9ec1904d4e76ae1cd1dc197e4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:09.487855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-812523fb-932f-494b-b1c7-322623cc03dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:09.494405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:09.534092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a574b55-769b-4f8c-af6c-2e5c2ee9fd30 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:09.537911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-6a574b55-769b-4f8c-af6c-2e5c2ee9fd30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:09.563785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a574b55-769b-4f8c-af6c-2e5c2ee9fd30 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:09.598696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:09.600366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:09.600617 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:09.602579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] PATH_INFO: `/v3/projects/7e0ee432504e4858b4287cbd2e318549` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:09.605973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0ee432504e4858b4287cbd2e318549)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:09.608620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:09.646525 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1212/2411] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:09.651290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812523fb-932f-494b-b1c7-322623cc03dc None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:09.652966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-812523fb-932f-494b-b1c7-322623cc03dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:09.669266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812523fb-932f-494b-b1c7-322623cc03dc None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:09.674338 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0ee432504e4858b4287cbd2e318549', 'request_id': 'req-ed443ece-6a65-4507-a878-9f4d007960e8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:09.684786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0ee432504e4858b4287cbd2e318549', 'request_id': 'req-ed443ece-6a65-4507-a878-9f4d007960e8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:09.692544 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0ee432504e4858b4287cbd2e318549', 'request_id': 'req-ed443ece-6a65-4507-a878-9f4d007960e8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:09.711798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] The token cache is being invalidate because project 7e0ee432504e4858b4287cbd2e318549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:56:09.712270 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:09.713104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ed443ece-6a65-4507-a878-9f4d007960e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:09.715303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1200/2412] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:56:09 2022] DELETE /identity/v3/projects/7e0ee432504e4858b4287cbd2e318549 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:56:09.722405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d5414-6f7f-4e55-9bd8-e24ca510f70a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:09.726316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d5414-6f7f-4e55-9bd8-e24ca510f70a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:09.726798 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d5414-6f7f-4e55-9bd8-e24ca510f70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:09.759880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-8c2d5414-6f7f-4e55-9bd8-e24ca510f70a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:09.761207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8c2d5414-6f7f-4e55-9bd8-e24ca510f70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:09.763233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeefb87-6637-41a3-9fbd-00c957f7754d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:09.764084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeefb87-6637-41a3-9fbd-00c957f7754d admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:09.764480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeefb87-6637-41a3-9fbd-00c957f7754d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:09.764808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeefb87-6637-41a3-9fbd-00c957f7754d admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:09.765610 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeefb87-6637-41a3-9fbd-00c957f7754d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:09.768263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeefb87-6637-41a3-9fbd-00c957f7754d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:09.780102 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1201/2413] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:09.781884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd4e01-ff85-4b3e-ac12-c7a479e4cab1 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:09.782345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd4e01-ff85-4b3e-ac12-c7a479e4cab1 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:09.782645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd4e01-ff85-4b3e-ac12-c7a479e4cab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:09.806591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a0fd4e01-ff85-4b3e-ac12-c7a479e4cab1 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:09.808069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a0fd4e01-ff85-4b3e-ac12-c7a479e4cab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:09.813011 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1213/2414] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:56:09 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:56:09.817571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeefb87-6637-41a3-9fbd-00c957f7754d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:09.819103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1202/2415] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:09.819822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ceeefb87-6637-41a3-9fbd-00c957f7754d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:09.823540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd4e01-ff85-4b3e-ac12-c7a479e4cab1 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:09.833489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd4e01-ff85-4b3e-ac12-c7a479e4cab1 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:09.897132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeefb87-6637-41a3-9fbd-00c957f7754d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:09.902992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23691b10-fd8e-4871-9f3c-59f71c3a0fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:09.904497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23691b10-fd8e-4871-9f3c-59f71c3a0fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:09.904780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23691b10-fd8e-4871-9f3c-59f71c3a0fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:09.905057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23691b10-fd8e-4871-9f3c-59f71c3a0fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:09.906014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23691b10-fd8e-4871-9f3c-59f71c3a0fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:09.908085 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23691b10-fd8e-4871-9f3c-59f71c3a0fd8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:09.911978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25813e-26a8-40ab-9c8b-8410d2179d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:09.912677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25813e-26a8-40ab-9c8b-8410d2179d0b admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:09.912959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25813e-26a8-40ab-9c8b-8410d2179d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:09.913234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25813e-26a8-40ab-9c8b-8410d2179d0b admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:09.914132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25813e-26a8-40ab-9c8b-8410d2179d0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:56:09.915389 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25813e-26a8-40ab-9c8b-8410d2179d0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:09.917038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25813e-26a8-40ab-9c8b-8410d2179d0b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:09.934745 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1214/2416] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:56:09 2022] POST /identity/v3/users => generated 299 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:56:09.938362 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23691b10-fd8e-4871-9f3c-59f71c3a0fd8 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:09.942550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1203/2417] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:56:09.943656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25813e-26a8-40ab-9c8b-8410d2179d0b admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:09.951922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23691b10-fd8e-4871-9f3c-59f71c3a0fd8 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:09.954369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25813e-26a8-40ab-9c8b-8410d2179d0b admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:09.958265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3d7e3-e493-4713-83b9-42d684c3d3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:09.959207 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3d7e3-e493-4713-83b9-42d684c3d3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:09.959565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3d7e3-e493-4713-83b9-42d684c3d3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:09.959906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3d7e3-e493-4713-83b9-42d684c3d3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:09.960104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f3df0-e714-413f-af90-17b1c274242d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:09.960770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f3df0-e714-413f-af90-17b1c274242d admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:09.961056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f3df0-e714-413f-af90-17b1c274242d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:09.961181 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3d7e3-e493-4713-83b9-42d684c3d3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:09.961328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f3df0-e714-413f-af90-17b1c274242d admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:09.962059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f3df0-e714-413f-af90-17b1c274242d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:09.964351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f3df0-e714-413f-af90-17b1c274242d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:09.964471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3d7e3-e493-4713-83b9-42d684c3d3a0 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:09.981326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1215/2418] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:56:09.985313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3d7e3-e493-4713-83b9-42d684c3d3a0 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:09.997199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1204/2419] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:56:09 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:56:09.999303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f3df0-e714-413f-af90-17b1c274242d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:09.999699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3d7e3-e493-4713-83b9-42d684c3d3a0 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:10.007697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:10.015083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:10.080857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cbad2-5a12-4a3f-81f7-be679e8f7f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:10.082009 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cbad2-5a12-4a3f-81f7-be679e8f7f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:10.082515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cbad2-5a12-4a3f-81f7-be679e8f7f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:10.082934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cbad2-5a12-4a3f-81f7-be679e8f7f0f admin admin] PATH_INFO: `/v3/projects/9b5bd7d318e349ccb7b4544e8353eac0/users/7da0114875c44e078e12c9d4264cd02c/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:10.094100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f3df0-e714-413f-af90-17b1c274242d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:10.100533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cbad2-5a12-4a3f-81f7-be679e8f7f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5bd7d318e349ccb7b4544e8353eac0, user_id=7da0114875c44e078e12c9d4264cd02c, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:10.100820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62528f2f-b72b-41ac-b4b1-3238bed34482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:10.101828 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62528f2f-b72b-41ac-b4b1-3238bed34482 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:10.102132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62528f2f-b72b-41ac-b4b1-3238bed34482 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:10.102562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62528f2f-b72b-41ac-b4b1-3238bed34482 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:10.103777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62528f2f-b72b-41ac-b4b1-3238bed34482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:10.104997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8cbad2-5a12-4a3f-81f7-be679e8f7f0f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:10.106237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62528f2f-b72b-41ac-b4b1-3238bed34482 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:10.127525 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1216/2420] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:56:09 2022] PUT /identity/v3/projects/9b5bd7d318e349ccb7b4544e8353eac0/users/7da0114875c44e078e12c9d4264cd02c/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:56:10.129791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cbad2-5a12-4a3f-81f7-be679e8f7f0f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:10.135471 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1205/2421] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:56:09 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:56:10.140312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62528f2f-b72b-41ac-b4b1-3238bed34482 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:10.151721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62528f2f-b72b-41ac-b4b1-3238bed34482 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:10.253549 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cbad2-5a12-4a3f-81f7-be679e8f7f0f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:10.259711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75031371-a631-419c-9097-dbcd225c0a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:10.260380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75031371-a631-419c-9097-dbcd225c0a75 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:10.260658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75031371-a631-419c-9097-dbcd225c0a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:10.260927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75031371-a631-419c-9097-dbcd225c0a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:10.261875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75031371-a631-419c-9097-dbcd225c0a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:10.262006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e00ec1-5f14-4ad5-a187-15c212e153eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:10.262647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e00ec1-5f14-4ad5-a187-15c212e153eb admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:10.262924 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e00ec1-5f14-4ad5-a187-15c212e153eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:10.263214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e00ec1-5f14-4ad5-a187-15c212e153eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:10.263367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75031371-a631-419c-9097-dbcd225c0a75 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:10.263935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e00ec1-5f14-4ad5-a187-15c212e153eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:10.265702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e00ec1-5f14-4ad5-a187-15c212e153eb admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:10.277165 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1206/2422] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:56:10.280363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1217/2423] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:56:10.281423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e00ec1-5f14-4ad5-a187-15c212e153eb admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:10.281537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75031371-a631-419c-9097-dbcd225c0a75 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:10.297230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e00ec1-5f14-4ad5-a187-15c212e153eb admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:10.304650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75031371-a631-419c-9097-dbcd225c0a75 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:10.310141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f2f8f-bb9a-42bf-ad1f-0819cb28a93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:10.310141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f2f8f-bb9a-42bf-ad1f-0819cb28a93b admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:10.310141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f2f8f-bb9a-42bf-ad1f-0819cb28a93b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:10.310507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f2f8f-bb9a-42bf-ad1f-0819cb28a93b admin admin] PATH_INFO: `/v3/projects/9b5bd7d318e349ccb7b4544e8353eac0/users/7da0114875c44e078e12c9d4264cd02c/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:10.312568 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f2f8f-bb9a-42bf-ad1f-0819cb28a93b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5bd7d318e349ccb7b4544e8353eac0, user_id=7da0114875c44e078e12c9d4264cd02c, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:10.317380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f2f8f-bb9a-42bf-ad1f-0819cb28a93b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:10.323346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fa983-1cfb-4f97-a5fe-40870e5ff6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:10.324050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fa983-1cfb-4f97-a5fe-40870e5ff6eb admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:10.324465 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fa983-1cfb-4f97-a5fe-40870e5ff6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:10.324655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fa983-1cfb-4f97-a5fe-40870e5ff6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:10.325483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fa983-1cfb-4f97-a5fe-40870e5ff6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:10.330820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fa983-1cfb-4f97-a5fe-40870e5ff6eb admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:10.337809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1207/2424] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:56:10 2022] PUT /identity/v3/projects/9b5bd7d318e349ccb7b4544e8353eac0/users/7da0114875c44e078e12c9d4264cd02c/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:56:10.345031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1218/2425] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:56:10.349637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f2f8f-bb9a-42bf-ad1f-0819cb28a93b admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:10.433309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f2f8f-bb9a-42bf-ad1f-0819cb28a93b admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:10.440212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b3be6-75ff-409e-980b-7356cdfd74c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:10.441223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b3be6-75ff-409e-980b-7356cdfd74c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:10.441564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b3be6-75ff-409e-980b-7356cdfd74c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:10.442077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b3be6-75ff-409e-980b-7356cdfd74c5 admin admin] PATH_INFO: `/v3/projects/bd1bec2c0a3d444da14743b95ad4e2b2/users/21aa14e7af2b4c2cb99f7ce07f80f88a/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:10.466168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b3be6-75ff-409e-980b-7356cdfd74c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1bec2c0a3d444da14743b95ad4e2b2, user_id=21aa14e7af2b4c2cb99f7ce07f80f88a, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:10.469904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b3be6-75ff-409e-980b-7356cdfd74c5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:10.491228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1208/2426] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:56:10 2022] PUT /identity/v3/projects/bd1bec2c0a3d444da14743b95ad4e2b2/users/21aa14e7af2b4c2cb99f7ce07f80f88a/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:56:10.505405 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836532d1-6617-4930-9c57-cc2b851cfda0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:10.507051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836532d1-6617-4930-9c57-cc2b851cfda0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:10.507589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836532d1-6617-4930-9c57-cc2b851cfda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:10.544422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-836532d1-6617-4930-9c57-cc2b851cfda0 None None] MFA Rules not processed for user `21aa14e7af2b4c2cb99f7ce07f80f88a`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:10.546319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-836532d1-6617-4930-9c57-cc2b851cfda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:10.662312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1219/2427] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:10.673780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b3be6-75ff-409e-980b-7356cdfd74c5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:10.676763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-808b3be6-75ff-409e-980b-7356cdfd74c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:10.682530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:10.817866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b3be6-75ff-409e-980b-7356cdfd74c5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:10.824006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b75976-060c-46e3-99f9-f539d4fcbb78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:10.824888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b75976-060c-46e3-99f9-f539d4fcbb78 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:10.825490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b75976-060c-46e3-99f9-f539d4fcbb78 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:10.826083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b75976-060c-46e3-99f9-f539d4fcbb78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:10.839398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b75976-060c-46e3-99f9-f539d4fcbb78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:10.845022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b75976-060c-46e3-99f9-f539d4fcbb78 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:10.869678 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1209/2428] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:56:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:56:15.494523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29c14f-da29-4c54-99df-328e7367a895 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:15.495042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29c14f-da29-4c54-99df-328e7367a895 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:15.495138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29c14f-da29-4c54-99df-328e7367a895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:15.512559 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:15.535226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-1f29c14f-da29-4c54-99df-328e7367a895 None None] MFA Rules not processed for user `7da0114875c44e078e12c9d4264cd02c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:15.537387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-1f29c14f-da29-4c54-99df-328e7367a895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:15.626271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1220/2429] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:15.635436 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b75976-060c-46e3-99f9-f539d4fcbb78 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:15.638381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-48b75976-060c-46e3-99f9-f539d4fcbb78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:15.757621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b75976-060c-46e3-99f9-f539d4fcbb78 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:15.762691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14340b-cbcc-4e31-8061-5c9eab30b863 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c022224b03a54ab489d95502c63a25dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:15.763491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14340b-cbcc-4e31-8061-5c9eab30b863 service glance] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:15.763823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14340b-cbcc-4e31-8061-5c9eab30b863 service glance] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:15.764279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14340b-cbcc-4e31-8061-5c9eab30b863 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:15.767566 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5a14340b-cbcc-4e31-8061-5c9eab30b863 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:15.854249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14340b-cbcc-4e31-8061-5c9eab30b863 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:15.855708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14340b-cbcc-4e31-8061-5c9eab30b863 service glance] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:15.866579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1210/2430] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:56:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:56:16.051885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3743e-7d92-427f-87e8-a1e550d993f7 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:16.052352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3743e-7d92-427f-87e8-a1e550d993f7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:16.052650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3743e-7d92-427f-87e8-a1e550d993f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:16.089348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-78e3743e-7d92-427f-87e8-a1e550d993f7 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:16.094836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-78e3743e-7d92-427f-87e8-a1e550d993f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:16.268456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1221/2431] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:56:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:16.289634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14340b-cbcc-4e31-8061-5c9eab30b863 service glance] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:16.291288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5a14340b-cbcc-4e31-8061-5c9eab30b863 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:16.343101 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:16.450642 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14340b-cbcc-4e31-8061-5c9eab30b863 service glance] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:16.460148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5965853-66b8-47c5-9547-16844b834e0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:16.460907 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5965853-66b8-47c5-9547-16844b834e0c service swift] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:16.461745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5965853-66b8-47c5-9547-16844b834e0c service swift] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:16.462028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5965853-66b8-47c5-9547-16844b834e0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:16.473620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5965853-66b8-47c5-9547-16844b834e0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:16.475664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5965853-66b8-47c5-9547-16844b834e0c service swift] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:16.486560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1211/2432] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:56:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:56:22.795755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3743e-7d92-427f-87e8-a1e550d993f7 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:22.885716 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3743e-7d92-427f-87e8-a1e550d993f7 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:22.893715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:22.894685 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:22.894969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:22.895351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] PATH_INFO: `/v3/users/21aa14e7af2b4c2cb99f7ce07f80f88a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:22.896579 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21aa14e7af2b4c2cb99f7ce07f80f88a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:22.898699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:22.967706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:22.977851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21aa14e7af2b4c2cb99f7ce07f80f88a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:22.989790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21aa14e7af2b4c2cb99f7ce07f80f88a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:22.991368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21aa14e7af2b4c2cb99f7ce07f80f88a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:23.014337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21aa14e7af2b4c2cb99f7ce07f80f88a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:23.015486 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21aa14e7af2b4c2cb99f7ce07f80f88a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:23.030221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1222/2433] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:56:22 2022] DELETE /identity/v3/users/21aa14e7af2b4c2cb99f7ce07f80f88a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:56:23.034683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56129f77-9fd0-4449-bf0f-4d842621822a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:23.036104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56129f77-9fd0-4449-bf0f-4d842621822a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:23.036436 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56129f77-9fd0-4449-bf0f-4d842621822a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:23.056939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-56129f77-9fd0-4449-bf0f-4d842621822a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:23.058203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-56129f77-9fd0-4449-bf0f-4d842621822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:23.177053 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1212/2434] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:23.184607 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:23.186532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:23.268595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469b875-8cd8-4eaf-a33e-0780bce144e1 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:23.275329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41cc55-592d-4103-a20b-93537921a1ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:23.276258 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41cc55-592d-4103-a20b-93537921a1ae service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:23.276659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41cc55-592d-4103-a20b-93537921a1ae service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:23.277002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41cc55-592d-4103-a20b-93537921a1ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:23.289746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41cc55-592d-4103-a20b-93537921a1ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:23.291238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41cc55-592d-4103-a20b-93537921a1ae service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:23.304627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1223/2435] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:56:23.625041 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56129f77-9fd0-4449-bf0f-4d842621822a None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:23.626632 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-56129f77-9fd0-4449-bf0f-4d842621822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:23.642882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56129f77-9fd0-4449-bf0f-4d842621822a None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:23.649534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:23.650922 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:23.651313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:23.651646 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] PATH_INFO: `/v3/projects/bd1bec2c0a3d444da14743b95ad4e2b2` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:23.653181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1bec2c0a3d444da14743b95ad4e2b2)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:23.655643 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:23.719851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1bec2c0a3d444da14743b95ad4e2b2', 'request_id': 'req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:23.731832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1bec2c0a3d444da14743b95ad4e2b2', 'request_id': 'req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:23.740602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1bec2c0a3d444da14743b95ad4e2b2', 'request_id': 'req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:23.755328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] The token cache is being invalidate because project bd1bec2c0a3d444da14743b95ad4e2b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:56:23.755764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:23.756441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-a5a9b406-020d-4b2a-bae5-33a4a50c9c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:56:23.758316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1213/2436] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:56:23 2022] DELETE /identity/v3/projects/bd1bec2c0a3d444da14743b95ad4e2b2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:56:26.725645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e2e0-ee20-4e18-9eeb-ff2cf5e6ba8f None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:26.726054 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e2e0-ee20-4e18-9eeb-ff2cf5e6ba8f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:26.726273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e2e0-ee20-4e18-9eeb-ff2cf5e6ba8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:26.744498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9c65e2e0-ee20-4e18-9eeb-ff2cf5e6ba8f None None] MFA Rules not processed for user `d4ef9ec1904d4e76ae1cd1dc197e4f41`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:26.745742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9c65e2e0-ee20-4e18-9eeb-ff2cf5e6ba8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:26.804773 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1224/2437] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:28.794811 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98f254-3c9f-4c71-98b7-9bf091dd92e6 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:28.795289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98f254-3c9f-4c71-98b7-9bf091dd92e6 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:28.795416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98f254-3c9f-4c71-98b7-9bf091dd92e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:28.818816 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ed98f254-3c9f-4c71-98b7-9bf091dd92e6 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:28.820067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ed98f254-3c9f-4c71-98b7-9bf091dd92e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:28.855215 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:28.902123 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1214/2438] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:56:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:56:28.909337 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e2e0-ee20-4e18-9eeb-ff2cf5e6ba8f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:56:28.911485 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9c65e2e0-ee20-4e18-9eeb-ff2cf5e6ba8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:29.010265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e2e0-ee20-4e18-9eeb-ff2cf5e6ba8f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:56:29.015346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bf1dc-13f2-42f7-84a1-5ee2b3233b79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:56:29.015998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bf1dc-13f2-42f7-84a1-5ee2b3233b79 service swift] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:29.016289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bf1dc-13f2-42f7-84a1-5ee2b3233b79 service swift] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:29.016550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bf1dc-13f2-42f7-84a1-5ee2b3233b79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:29.021874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:29.025881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bf1dc-13f2-42f7-84a1-5ee2b3233b79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:56:29.028075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bf1dc-13f2-42f7-84a1-5ee2b3233b79 service swift] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:56:29.041758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1225/2439] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:56:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 19:56:38.887041 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=9 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:39.038552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:56:57.756435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ca130-a32a-4a66-8c07-abf6295ad860 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:56:57.756868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ca130-a32a-4a66-8c07-abf6295ad860 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:56:57.756995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ca130-a32a-4a66-8c07-abf6295ad860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:56:57.777234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-140ca130-a32a-4a66-8c07-abf6295ad860 None None] MFA Rules not processed for user `3307be3159654f55be46e75c82584bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:56:57.778483 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-140ca130-a32a-4a66-8c07-abf6295ad860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:56:57.840456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1215/2440] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:56:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:57:01.275128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bf1dc-13f2-42f7-84a1-5ee2b3233b79 service swift] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:57:01.276532 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-602bf1dc-13f2-42f7-84a1-5ee2b3233b79 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:01.332264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bf1dc-13f2-42f7-84a1-5ee2b3233b79 service swift] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:57:01.336880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fbc82-eb94-4551-a272-cd3515ec4038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:57:01.337518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fbc82-eb94-4551-a272-cd3515ec4038 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:01.337827 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fbc82-eb94-4551-a272-cd3515ec4038 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:01.338100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fbc82-eb94-4551-a272-cd3515ec4038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:01.339880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-111fbc82-eb94-4551-a272-cd3515ec4038 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:01.403571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fbc82-eb94-4551-a272-cd3515ec4038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:57:01.405062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111fbc82-eb94-4551-a272-cd3515ec4038 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:57:01.414775 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1226/2441] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:57:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:57:02.283495 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea359da-4865-4e8f-8b05-2a1f4f162eb1 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:02.284027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea359da-4865-4e8f-8b05-2a1f4f162eb1 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:02.284027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea359da-4865-4e8f-8b05-2a1f4f162eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:02.303447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-9ea359da-4865-4e8f-8b05-2a1f4f162eb1 None None] MFA Rules not processed for user `0267350339c44000bcc49906cfa8f0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:57:02.304598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9ea359da-4865-4e8f-8b05-2a1f4f162eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:02.364258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1216/2442] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:57:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:57:11.410549 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=50 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:57:12.356794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=50 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:57:18.596868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c6914-2b3c-4a22-9f3f-5585d20805e6 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:18.597375 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c6914-2b3c-4a22-9f3f-5585d20805e6 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:18.597472 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c6914-2b3c-4a22-9f3f-5585d20805e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:18.616269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-b85c6914-2b3c-4a22-9f3f-5585d20805e6 None None] MFA Rules not processed for user `3198a63f6d1a4cdd88d76189658e79c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:57:18.617451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b85c6914-2b3c-4a22-9f3f-5585d20805e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:18.671905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1227/2443] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:57:28.665259 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=25 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:57:35.252959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65c2ae-1c91-414b-8f39-e3b84ebfb967 None None] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:35.253519 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65c2ae-1c91-414b-8f39-e3b84ebfb967 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:35.253519 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65c2ae-1c91-414b-8f39-e3b84ebfb967 None None] PATH_INFO: `/` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:35.255488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1217/2444] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:57:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:57:35.261519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df568529-8d4c-4055-9891-5c25ec373a1a None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:35.262052 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df568529-8d4c-4055-9891-5c25ec373a1a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:35.262165 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df568529-8d4c-4055-9891-5c25ec373a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:35.281317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-df568529-8d4c-4055-9891-5c25ec373a1a None None] MFA Rules not processed for user `370f1c0f7af04a56bdf8fa452550380a`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:57:35.282769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-df568529-8d4c-4055-9891-5c25ec373a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:35.289859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1228/2445] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:57:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:57:35.304653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65c2ae-1c91-414b-8f39-e3b84ebfb967 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:57:35.307872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-cd65c2ae-1c91-414b-8f39-e3b84ebfb967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:35.323580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65c2ae-1c91-414b-8f39-e3b84ebfb967 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:57:35.327371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c96b7c-b262-44b7-bfb4-41e8ec6b0c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:57:35.328090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c96b7c-b262-44b7-bfb4-41e8ec6b0c7c service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:35.329041 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c96b7c-b262-44b7-bfb4-41e8ec6b0c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:35.329394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c96b7c-b262-44b7-bfb4-41e8ec6b0c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:35.338083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c96b7c-b262-44b7-bfb4-41e8ec6b0c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:57:35.339603 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c96b7c-b262-44b7-bfb4-41e8ec6b0c7c service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:57:35.347642 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1218/2446] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:57:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:57:35.457602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d52da9-e3dc-42e7-8c55-0846a2398e6f None None] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:35.458039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d52da9-e3dc-42e7-8c55-0846a2398e6f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:35.458325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d52da9-e3dc-42e7-8c55-0846a2398e6f None None] PATH_INFO: `/` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:35.459734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1229/2447] 10.222.0.66 () {54 vars in 921 bytes} [Fri Jul 1 19:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 19:57:35.465227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe54b6-0011-4745-8f25-c53a66aa0c55 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:35.465604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe54b6-0011-4745-8f25-c53a66aa0c55 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:35.465964 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe54b6-0011-4745-8f25-c53a66aa0c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:35.539410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-46fe54b6-0011-4745-8f25-c53a66aa0c55 None None] MFA Rules not processed for user `ad06f31859f9413f9db6039ac2abf982`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:57:35.540824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-46fe54b6-0011-4745-8f25-c53a66aa0c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:35.632840 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1219/2448] 10.222.0.66 () {58 vars in 1018 bytes} [Fri Jul 1 19:57:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:57:35.643368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d52da9-e3dc-42e7-8c55-0846a2398e6f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:57:35.654904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d52da9-e3dc-42e7-8c55-0846a2398e6f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:57:35.664292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546520e-85ea-4aa6-8b82-95686c40f83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:57:35.664869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546520e-85ea-4aa6-8b82-95686c40f83d service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:35.665144 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546520e-85ea-4aa6-8b82-95686c40f83d service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:35.665412 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546520e-85ea-4aa6-8b82-95686c40f83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:35.676528 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546520e-85ea-4aa6-8b82-95686c40f83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:57:35.678032 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546520e-85ea-4aa6-8b82-95686c40f83d service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:57:35.688874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1230/2449] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:57:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:57:42.423912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe54b6-0011-4745-8f25-c53a66aa0c55 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:57:42.434032 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe54b6-0011-4745-8f25-c53a66aa0c55 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:57:42.437440 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca6209-5484-4bec-b47e-3c175041e3ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:57:42.438099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca6209-5484-4bec-b47e-3c175041e3ba service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:42.438379 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca6209-5484-4bec-b47e-3c175041e3ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:42.438665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca6209-5484-4bec-b47e-3c175041e3ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:42.440376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f5ca6209-5484-4bec-b47e-3c175041e3ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:42.454445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca6209-5484-4bec-b47e-3c175041e3ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:57:42.455928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca6209-5484-4bec-b47e-3c175041e3ba service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:57:42.467258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1220/2450] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:57:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:57:43.404202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546520e-85ea-4aa6-8b82-95686c40f83d service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:57:43.413065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546520e-85ea-4aa6-8b82-95686c40f83d service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:57:43.416590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948666b8-c405-4696-94c7-8e8590f5d0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:57:43.417216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948666b8-c405-4696-94c7-8e8590f5d0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:43.417493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948666b8-c405-4696-94c7-8e8590f5d0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:43.417801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948666b8-c405-4696-94c7-8e8590f5d0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:43.419656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-948666b8-c405-4696-94c7-8e8590f5d0d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:43.433103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948666b8-c405-4696-94c7-8e8590f5d0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:57:43.434554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948666b8-c405-4696-94c7-8e8590f5d0d1 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:57:43.446098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1231/2451] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:57:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:57:52.462451 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=58 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:57:53.440598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=15 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:57:54.291255 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca6209-5484-4bec-b47e-3c175041e3ba service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:57:54.292758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f5ca6209-5484-4bec-b47e-3c175041e3ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:54.312547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca6209-5484-4bec-b47e-3c175041e3ba service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:57:54.317825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390fe48-b0cc-4aba-8f77-52e25791ecab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:57:54.318488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390fe48-b0cc-4aba-8f77-52e25791ecab service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:57:54.318769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390fe48-b0cc-4aba-8f77-52e25791ecab service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:57:54.319042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390fe48-b0cc-4aba-8f77-52e25791ecab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:57:54.320824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5390fe48-b0cc-4aba-8f77-52e25791ecab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:57:54.336698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390fe48-b0cc-4aba-8f77-52e25791ecab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:57:54.338333 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390fe48-b0cc-4aba-8f77-52e25791ecab service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:57:54.347720 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1221/2452] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:57:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:03.553861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948666b8-c405-4696-94c7-8e8590f5d0d1 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:03.555269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-948666b8-c405-4696-94c7-8e8590f5d0d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:03.569466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948666b8-c405-4696-94c7-8e8590f5d0d1 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:03.575013 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4b874-1fa2-44e1-afc7-cc2036d09722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:03.575631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4b874-1fa2-44e1-afc7-cc2036d09722 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:03.575913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4b874-1fa2-44e1-afc7-cc2036d09722 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:03.576187 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4b874-1fa2-44e1-afc7-cc2036d09722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:03.578045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e4c4b874-1fa2-44e1-afc7-cc2036d09722 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:03.589317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4b874-1fa2-44e1-afc7-cc2036d09722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:03.591058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4b874-1fa2-44e1-afc7-cc2036d09722 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:03.600351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1232/2453] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:04.343281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=6 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:58:10.834962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390fe48-b0cc-4aba-8f77-52e25791ecab service nova] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:10.844156 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390fe48-b0cc-4aba-8f77-52e25791ecab service nova] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:10.847729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492f623-0a0b-4522-a7a9-0e7aab8d3350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:10.848930 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492f623-0a0b-4522-a7a9-0e7aab8d3350 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:10.849210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492f623-0a0b-4522-a7a9-0e7aab8d3350 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:10.849480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492f623-0a0b-4522-a7a9-0e7aab8d3350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:10.851275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e492f623-0a0b-4522-a7a9-0e7aab8d3350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:10.918216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492f623-0a0b-4522-a7a9-0e7aab8d3350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:10.919654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492f623-0a0b-4522-a7a9-0e7aab8d3350 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:10.930430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1222/2454] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:13.595975 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=6 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:58:14.045498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4b874-1fa2-44e1-afc7-cc2036d09722 service nova] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:14.047945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e4c4b874-1fa2-44e1-afc7-cc2036d09722 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:14.062089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4b874-1fa2-44e1-afc7-cc2036d09722 service nova] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:14.067037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:14.067956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.068230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.068500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] PATH_INFO: `/v3/users/0267350339c44000bcc49906cfa8f0a6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.069718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0267350339c44000bcc49906cfa8f0a6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:14.071634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:14.117497 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0267350339c44000bcc49906cfa8f0a6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:14.125272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0267350339c44000bcc49906cfa8f0a6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:14.125911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0267350339c44000bcc49906cfa8f0a6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:14.138361 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0267350339c44000bcc49906cfa8f0a6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:14.138992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-98e67a18-738b-4378-8cbc-3bee72f1fa5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0267350339c44000bcc49906cfa8f0a6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:14.151291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1233/2455] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:58:14 2022] DELETE /identity/v3/users/0267350339c44000bcc49906cfa8f0a6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:58:14.327160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492f623-0a0b-4522-a7a9-0e7aab8d3350 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:14.337960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492f623-0a0b-4522-a7a9-0e7aab8d3350 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:14.403955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:14.404735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.405015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.405288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] PATH_INFO: `/v3/projects/0ba0ddfd3efd41ee9687efe014c40bf0` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.406496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba0ddfd3efd41ee9687efe014c40bf0)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:14.408394 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:14.453694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba0ddfd3efd41ee9687efe014c40bf0', 'request_id': 'req-abc3a062-325b-47d0-9426-81a77f03991e'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:14.462220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba0ddfd3efd41ee9687efe014c40bf0', 'request_id': 'req-abc3a062-325b-47d0-9426-81a77f03991e'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:14.467967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba0ddfd3efd41ee9687efe014c40bf0', 'request_id': 'req-abc3a062-325b-47d0-9426-81a77f03991e'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:14.479765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] The token cache is being invalidate because project 0ba0ddfd3efd41ee9687efe014c40bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:58:14.480051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:14.480696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-abc3a062-325b-47d0-9426-81a77f03991e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:14.482275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1223/2456] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:58:14 2022] DELETE /identity/v3/projects/0ba0ddfd3efd41ee9687efe014c40bf0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:58:14.487330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be40834-a1a4-456c-acbd-0c09337c006e None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.487633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be40834-a1a4-456c-acbd-0c09337c006e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.487903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be40834-a1a4-456c-acbd-0c09337c006e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.511476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5be40834-a1a4-456c-acbd-0c09337c006e None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:58:14.512737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5be40834-a1a4-456c-acbd-0c09337c006e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:14.575363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1234/2457] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:58:14.583555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135caba-c477-4f6e-9106-f92368f50600 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.583943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135caba-c477-4f6e-9106-f92368f50600 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.584291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135caba-c477-4f6e-9106-f92368f50600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.608391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-8135caba-c477-4f6e-9106-f92368f50600 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:58:14.609683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8135caba-c477-4f6e-9106-f92368f50600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:14.618421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1224/2458] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:58:14.623610 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be40834-a1a4-456c-acbd-0c09337c006e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:14.634247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be40834-a1a4-456c-acbd-0c09337c006e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:14.638719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f05cf-a3b6-4899-b601-7e298641541f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:14.639359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f05cf-a3b6-4899-b601-7e298641541f admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.639634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f05cf-a3b6-4899-b601-7e298641541f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.639900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f05cf-a3b6-4899-b601-7e298641541f admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.640609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f05cf-a3b6-4899-b601-7e298641541f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:58:14.640906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f05cf-a3b6-4899-b601-7e298641541f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:14.642524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f05cf-a3b6-4899-b601-7e298641541f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:14.655054 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1235/2459] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:58:14.659150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135caba-c477-4f6e-9106-f92368f50600 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:14.669191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135caba-c477-4f6e-9106-f92368f50600 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:14.674001 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebef34-1d3c-47a7-b773-6589196712a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:14.674641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebef34-1d3c-47a7-b773-6589196712a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.674921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebef34-1d3c-47a7-b773-6589196712a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.675193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebef34-1d3c-47a7-b773-6589196712a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.675904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebef34-1d3c-47a7-b773-6589196712a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:14.678383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebef34-1d3c-47a7-b773-6589196712a5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:14.691909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:58:14.705009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1225/2460] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:58:14 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:58:14.709794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f05cf-a3b6-4899-b601-7e298641541f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:14.719781 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f05cf-a3b6-4899-b601-7e298641541f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:14.776981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1b3f8-8a25-474b-8168-2e17b50b2050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:14.777704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1b3f8-8a25-474b-8168-2e17b50b2050 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.777988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1b3f8-8a25-474b-8168-2e17b50b2050 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.778261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1b3f8-8a25-474b-8168-2e17b50b2050 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.778965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1b3f8-8a25-474b-8168-2e17b50b2050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:14.780873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1b3f8-8a25-474b-8168-2e17b50b2050 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:14.788387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:58:14.802626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1236/2461] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:58:14 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:58:14.806783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebef34-1d3c-47a7-b773-6589196712a5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:14.817564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebef34-1d3c-47a7-b773-6589196712a5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:14.822182 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0960b-6993-4f16-a217-433686dbd031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:14.822834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0960b-6993-4f16-a217-433686dbd031 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.823118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0960b-6993-4f16-a217-433686dbd031 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.823395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0960b-6993-4f16-a217-433686dbd031 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.824120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0960b-6993-4f16-a217-433686dbd031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:14.825760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f0960b-6993-4f16-a217-433686dbd031 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:14.837899 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1226/2462] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:58:14.842249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1b3f8-8a25-474b-8168-2e17b50b2050 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:14.852423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1b3f8-8a25-474b-8168-2e17b50b2050 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:14.857024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1c060-41ba-4c9f-ab52-a1bc69f4fbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:14.857737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1c060-41ba-4c9f-ab52-a1bc69f4fbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.858019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1c060-41ba-4c9f-ab52-a1bc69f4fbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.858294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1c060-41ba-4c9f-ab52-a1bc69f4fbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.859058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1c060-41ba-4c9f-ab52-a1bc69f4fbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:14.860655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1c060-41ba-4c9f-ab52-a1bc69f4fbf7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:14.870692 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1237/2463] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:58:14.875296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0960b-6993-4f16-a217-433686dbd031 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:14.885851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0960b-6993-4f16-a217-433686dbd031 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:14.890476 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70add7-af6a-44e7-8638-558dca7fc49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:14.891349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70add7-af6a-44e7-8638-558dca7fc49c admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.891627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70add7-af6a-44e7-8638-558dca7fc49c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.891900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70add7-af6a-44e7-8638-558dca7fc49c admin admin] PATH_INFO: `/v3/projects/983da191c3d44e608e40f289e26d3603/users/8f2b9a6934d34f58a09a4cefb154371c/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.907086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70add7-af6a-44e7-8638-558dca7fc49c admin admin] RBAC: Authorizing `identity:create_grant(project_id=983da191c3d44e608e40f289e26d3603, user_id=8f2b9a6934d34f58a09a4cefb154371c, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:14.909699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70add7-af6a-44e7-8638-558dca7fc49c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:14.922926 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1227/2464] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:58:14 2022] PUT /identity/v3/projects/983da191c3d44e608e40f289e26d3603/users/8f2b9a6934d34f58a09a4cefb154371c/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:58:14.928724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f8b5d-90a0-4ee6-a9b1-a99f29c0a2d0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:14.929015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f8b5d-90a0-4ee6-a9b1-a99f29c0a2d0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:14.929294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f8b5d-90a0-4ee6-a9b1-a99f29c0a2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:14.946719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-4c7f8b5d-90a0-4ee6-a9b1-a99f29c0a2d0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:58:14.947911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4c7f8b5d-90a0-4ee6-a9b1-a99f29c0a2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:15.007889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1238/2465] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:58:15.017504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70add7-af6a-44e7-8638-558dca7fc49c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:15.019202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1d70add7-af6a-44e7-8638-558dca7fc49c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:15.076738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70add7-af6a-44e7-8638-558dca7fc49c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:15.080294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d309d5-fd9d-4c70-b54f-85db444d0e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:15.080921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d309d5-fd9d-4c70-b54f-85db444d0e84 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:15.081308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d309d5-fd9d-4c70-b54f-85db444d0e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:15.081604 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d309d5-fd9d-4c70-b54f-85db444d0e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:15.091322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d309d5-fd9d-4c70-b54f-85db444d0e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:15.092759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d309d5-fd9d-4c70-b54f-85db444d0e84 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:15.102794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1228/2466] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:18.384120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f8b5d-90a0-4ee6-a9b1-a99f29c0a2d0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:18.385665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4c7f8b5d-90a0-4ee6-a9b1-a99f29c0a2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:18.498763 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f8b5d-90a0-4ee6-a9b1-a99f29c0a2d0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:18.519386 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19083119-9e74-4a23-9f34-aeea76766d41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:18.520636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19083119-9e74-4a23-9f34-aeea76766d41 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:18.521281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19083119-9e74-4a23-9f34-aeea76766d41 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:18.521281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19083119-9e74-4a23-9f34-aeea76766d41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:18.523902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-19083119-9e74-4a23-9f34-aeea76766d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:18.601253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19083119-9e74-4a23-9f34-aeea76766d41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:18.603273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19083119-9e74-4a23-9f34-aeea76766d41 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:18.617769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1239/2467] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:19.182681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43795630-a19f-4307-b121-23bac44397b4 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:19.183155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43795630-a19f-4307-b121-23bac44397b4 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:19.183426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43795630-a19f-4307-b121-23bac44397b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:19.213758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-43795630-a19f-4307-b121-23bac44397b4 None None] MFA Rules not processed for user `8f2b9a6934d34f58a09a4cefb154371c`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:58:19.215231 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-43795630-a19f-4307-b121-23bac44397b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:19.307998 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1229/2468] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:58:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:58:19.323608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19083119-9e74-4a23-9f34-aeea76766d41 service nova] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:19.325033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-19083119-9e74-4a23-9f34-aeea76766d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:19.354650 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19083119-9e74-4a23-9f34-aeea76766d41 service nova] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:19.365800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d2373-8d7c-4a66-a1f9-3d21e5993ef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:19.367156 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d2373-8d7c-4a66-a1f9-3d21e5993ef5 service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:19.367772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d2373-8d7c-4a66-a1f9-3d21e5993ef5 service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:19.368334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d2373-8d7c-4a66-a1f9-3d21e5993ef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:19.386086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d2373-8d7c-4a66-a1f9-3d21e5993ef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:19.387750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d2373-8d7c-4a66-a1f9-3d21e5993ef5 service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:19.411706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1240/2469] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:20.373095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43795630-a19f-4307-b121-23bac44397b4 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:20.400212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43795630-a19f-4307-b121-23bac44397b4 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:20.416448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee0ebb-7d6c-470e-b19c-57bade981d3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:20.417138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee0ebb-7d6c-470e-b19c-57bade981d3f service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:20.417419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee0ebb-7d6c-470e-b19c-57bade981d3f service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:20.417820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee0ebb-7d6c-470e-b19c-57bade981d3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:20.422773 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f0ee0ebb-7d6c-470e-b19c-57bade981d3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:20.451081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee0ebb-7d6c-470e-b19c-57bade981d3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:20.452975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee0ebb-7d6c-470e-b19c-57bade981d3f service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:20.476897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1230/2470] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:21.889253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c248c-e9ac-4b40-9ea9-5a9cd9242233 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:21.889715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c248c-e9ac-4b40-9ea9-5a9cd9242233 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:21.889886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c248c-e9ac-4b40-9ea9-5a9cd9242233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:21.897143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:58:21.912509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-762c248c-e9ac-4b40-9ea9-5a9cd9242233 None None] MFA Rules not processed for user `8f2b9a6934d34f58a09a4cefb154371c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:58:21.913747 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-762c248c-e9ac-4b40-9ea9-5a9cd9242233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:21.920740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1241/2471] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:58:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:58:22.106971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ae5d9-2fde-489d-95be-da290f7e2a7e None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:22.107456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ae5d9-2fde-489d-95be-da290f7e2a7e None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:22.107562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ae5d9-2fde-489d-95be-da290f7e2a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:22.126153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-429ae5d9-2fde-489d-95be-da290f7e2a7e None None] MFA Rules not processed for user `80267c31f3fc4b3fbc8dc7ed03326eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:58:22.127383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-429ae5d9-2fde-489d-95be-da290f7e2a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:22.163355 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:58:22.196190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1231/2472] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:58:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:58:22.276153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c248c-e9ac-4b40-9ea9-5a9cd9242233 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:22.279540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-762c248c-e9ac-4b40-9ea9-5a9cd9242233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:22.297183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c248c-e9ac-4b40-9ea9-5a9cd9242233 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:22.303154 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4e1e6-d9fc-4db5-9f88-3f8275aafb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:22.303878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4e1e6-d9fc-4db5-9f88-3f8275aafb8c service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:22.304220 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4e1e6-d9fc-4db5-9f88-3f8275aafb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:22.304560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4e1e6-d9fc-4db5-9f88-3f8275aafb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:22.306469 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ccd4e1e6-d9fc-4db5-9f88-3f8275aafb8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:22.326942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4e1e6-d9fc-4db5-9f88-3f8275aafb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:22.330153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4e1e6-d9fc-4db5-9f88-3f8275aafb8c service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:22.344728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1242/2473] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:22.774586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ae5d9-2fde-489d-95be-da290f7e2a7e None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:22.776519 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-429ae5d9-2fde-489d-95be-da290f7e2a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:22.845144 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4e1e6-d9fc-4db5-9f88-3f8275aafb8c service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:22.857762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4e1e6-d9fc-4db5-9f88-3f8275aafb8c service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:22.868020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecefdf-e7bb-41df-a8ae-59ae33bf2302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:22.869231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecefdf-e7bb-41df-a8ae-59ae33bf2302 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:22.869754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecefdf-e7bb-41df-a8ae-59ae33bf2302 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:22.870257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecefdf-e7bb-41df-a8ae-59ae33bf2302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:22.877524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-39ecefdf-e7bb-41df-a8ae-59ae33bf2302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:22.914179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecefdf-e7bb-41df-a8ae-59ae33bf2302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:22.915701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecefdf-e7bb-41df-a8ae-59ae33bf2302 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:22.925227 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1243/2474] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:58:22.936694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ae5d9-2fde-489d-95be-da290f7e2a7e None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:22.947465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac3709-f551-4543-928e-e5de3ec778e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:22.948302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac3709-f551-4543-928e-e5de3ec778e0 service placement] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:22.948631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac3709-f551-4543-928e-e5de3ec778e0 service placement] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:22.948961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac3709-f551-4543-928e-e5de3ec778e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:22.951675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a3ac3709-f551-4543-928e-e5de3ec778e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:22.970226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac3709-f551-4543-928e-e5de3ec778e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:22.972381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac3709-f551-4543-928e-e5de3ec778e0 service placement] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:22.984848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1232/2475] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:25.577376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecefdf-e7bb-41df-a8ae-59ae33bf2302 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:25.595275 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecefdf-e7bb-41df-a8ae-59ae33bf2302 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:25.602321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1344b4c-6e8b-49fe-a4b9-27762b2af1be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:25.602992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1344b4c-6e8b-49fe-a4b9-27762b2af1be service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:25.603438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1344b4c-6e8b-49fe-a4b9-27762b2af1be service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:25.604224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1344b4c-6e8b-49fe-a4b9-27762b2af1be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:25.608983 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f1344b4c-6e8b-49fe-a4b9-27762b2af1be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:25.623964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1344b4c-6e8b-49fe-a4b9-27762b2af1be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:25.625425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1344b4c-6e8b-49fe-a4b9-27762b2af1be service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:25.637205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1244/2476] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:26.983939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac3709-f551-4543-928e-e5de3ec778e0 service placement] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:27.000248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac3709-f551-4543-928e-e5de3ec778e0 service placement] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:27.006771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a6532-f01c-4bca-bfb5-fbcf6d0bfd5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:27.007517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a6532-f01c-4bca-bfb5-fbcf6d0bfd5e service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:27.007803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a6532-f01c-4bca-bfb5-fbcf6d0bfd5e service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:27.008079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a6532-f01c-4bca-bfb5-fbcf6d0bfd5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:27.009967 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d29a6532-f01c-4bca-bfb5-fbcf6d0bfd5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:27.023960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a6532-f01c-4bca-bfb5-fbcf6d0bfd5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:27.025532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a6532-f01c-4bca-bfb5-fbcf6d0bfd5e service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:27.037349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1233/2477] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:58:27.959185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e288da3-aee0-4c8a-bfc3-3785a545ab6b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:27.959715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e288da3-aee0-4c8a-bfc3-3785a545ab6b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:27.960153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e288da3-aee0-4c8a-bfc3-3785a545ab6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:28.001819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-2e288da3-aee0-4c8a-bfc3-3785a545ab6b None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:58:28.003225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2e288da3-aee0-4c8a-bfc3-3785a545ab6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:28.088216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1245/2478] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:58:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:58:28.147558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a6532-f01c-4bca-bfb5-fbcf6d0bfd5e service nova] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:28.149166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d29a6532-f01c-4bca-bfb5-fbcf6d0bfd5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:28.245978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a6532-f01c-4bca-bfb5-fbcf6d0bfd5e service nova] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:28.251700 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f4a94-13bb-43c6-99ae-7aa5885ac067 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:28.252536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f4a94-13bb-43c6-99ae-7aa5885ac067 service swift] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:28.253035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f4a94-13bb-43c6-99ae-7aa5885ac067 service swift] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:28.253599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f4a94-13bb-43c6-99ae-7aa5885ac067 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:28.267383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f4a94-13bb-43c6-99ae-7aa5885ac067 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:28.269695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f4a94-13bb-43c6-99ae-7aa5885ac067 service swift] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:28.280801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1234/2479] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:58:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:58:30.439988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e288da3-aee0-4c8a-bfc3-3785a545ab6b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:30.470920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e288da3-aee0-4c8a-bfc3-3785a545ab6b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:30.495452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d229b-846b-4d58-96b5-187799f091bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:30.497242 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d229b-846b-4d58-96b5-187799f091bd service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:30.497622 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d229b-846b-4d58-96b5-187799f091bd service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:30.497964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d229b-846b-4d58-96b5-187799f091bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:30.500256 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-fd5d229b-846b-4d58-96b5-187799f091bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:30.523033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d229b-846b-4d58-96b5-187799f091bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:30.525311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d229b-846b-4d58-96b5-187799f091bd service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:30.538973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1246/2480] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:58:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:58:38.278822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=77 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:58:40.532387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=51 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:58:46.844239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a2f9-b207-42e4-b322-3d0b5b2873e9 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:46.844724 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a2f9-b207-42e4-b322-3d0b5b2873e9 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:46.845067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93a2f9-b207-42e4-b322-3d0b5b2873e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:46.863580 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ab93a2f9-b207-42e4-b322-3d0b5b2873e9 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:58:46.864888 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ab93a2f9-b207-42e4-b322-3d0b5b2873e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:46.872115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1235/2481] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 1 19:58:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:58:46.885682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d229b-846b-4d58-96b5-187799f091bd service nova] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:46.898283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d229b-846b-4d58-96b5-187799f091bd service nova] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:46.903084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308258d-673b-45a8-b5b4-def270522074 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:46.903721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308258d-673b-45a8-b5b4-def270522074 service swift] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:46.904000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308258d-673b-45a8-b5b4-def270522074 service swift] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:46.904270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308258d-673b-45a8-b5b4-def270522074 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:46.913746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308258d-673b-45a8-b5b4-def270522074 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:46.915326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308258d-673b-45a8-b5b4-def270522074 service swift] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:46.924033 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1247/2482] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:58:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:58:49.175700 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a882b6-b041-4766-b8b4-c085a4a127df None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:49.176154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a882b6-b041-4766-b8b4-c085a4a127df None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:49.176356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a882b6-b041-4766-b8b4-c085a4a127df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:49.196516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-32a882b6-b041-4766-b8b4-c085a4a127df None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:58:49.197750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-32a882b6-b041-4766-b8b4-c085a4a127df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:49.207216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1236/2483] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 19:58:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:58:49.220931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308258d-673b-45a8-b5b4-def270522074 service swift] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:49.232851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308258d-673b-45a8-b5b4-def270522074 service swift] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:49.239082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c258c74-d16b-43dc-b56b-57499fe9b785 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:49.239733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c258c74-d16b-43dc-b56b-57499fe9b785 service swift] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:49.240017 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c258c74-d16b-43dc-b56b-57499fe9b785 service swift] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:49.240308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c258c74-d16b-43dc-b56b-57499fe9b785 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:49.249164 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c258c74-d16b-43dc-b56b-57499fe9b785 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:49.250895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c258c74-d16b-43dc-b56b-57499fe9b785 service swift] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:49.259939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1248/2484] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:58:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:58:57.303115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a882b6-b041-4766-b8b4-c085a4a127df None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:57.304666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-32a882b6-b041-4766-b8b4-c085a4a127df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:57.329693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a882b6-b041-4766-b8b4-c085a4a127df None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:57.340963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:57.344818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:57.345097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:57.345380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] PATH_INFO: `/v3/users/d4ef9ec1904d4e76ae1cd1dc197e4f41` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:57.348368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ef9ec1904d4e76ae1cd1dc197e4f41)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:57.354570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:57.429548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ef9ec1904d4e76ae1cd1dc197e4f41'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:57.448555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ef9ec1904d4e76ae1cd1dc197e4f41'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:57.458885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ef9ec1904d4e76ae1cd1dc197e4f41'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:57.466506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ef9ec1904d4e76ae1cd1dc197e4f41'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:57.470023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ef9ec1904d4e76ae1cd1dc197e4f41'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:57.477470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1237/2485] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:58:57 2022] DELETE /identity/v3/users/d4ef9ec1904d4e76ae1cd1dc197e4f41 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:58:57.692039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c258c74-d16b-43dc-b56b-57499fe9b785 service swift] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:57.709655 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c258c74-d16b-43dc-b56b-57499fe9b785 service swift] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:57.785945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:57.787132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:57.787413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:57.787686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] PATH_INFO: `/v3/projects/e33de0b0c3244c78a86fbfda1c9fbdd6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:57.788898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e33de0b0c3244c78a86fbfda1c9fbdd6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:57.790857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:57.850647 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e33de0b0c3244c78a86fbfda1c9fbdd6', 'request_id': 'req-8fde31b4-f222-40aa-8f02-f695bb95e8e3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:57.858514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e33de0b0c3244c78a86fbfda1c9fbdd6', 'request_id': 'req-8fde31b4-f222-40aa-8f02-f695bb95e8e3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:57.866351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e33de0b0c3244c78a86fbfda1c9fbdd6', 'request_id': 'req-8fde31b4-f222-40aa-8f02-f695bb95e8e3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:57.877531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] The token cache is being invalidate because project e33de0b0c3244c78a86fbfda1c9fbdd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:58:57.878676 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:57.881755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:57.884918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1249/2486] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:58:57 2022] DELETE /identity/v3/projects/e33de0b0c3244c78a86fbfda1c9fbdd6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:58:57.888802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:57.890477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:57.905023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f089b-6426-4f50-8613-60cd8e4bbf2f admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:57.979227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:57.980112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:57.980596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:57.981047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] PATH_INFO: `/v3/users/7da0114875c44e078e12c9d4264cd02c` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:57.982500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da0114875c44e078e12c9d4264cd02c)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:57.984602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:58.049601 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da0114875c44e078e12c9d4264cd02c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:58.066258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:58:58.071571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da0114875c44e078e12c9d4264cd02c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:58.079597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da0114875c44e078e12c9d4264cd02c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:58.087692 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da0114875c44e078e12c9d4264cd02c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:58.088371 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da0114875c44e078e12c9d4264cd02c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:58.094827 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1238/2487] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:58:57 2022] DELETE /identity/v3/users/7da0114875c44e078e12c9d4264cd02c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:58:58.285393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:58:58.291887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:58:58.322326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde31b4-f222-40aa-8f02-f695bb95e8e3 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:58:58.399249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:58:58.400388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:58:58.400866 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:58:58.401138 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] PATH_INFO: `/v3/projects/9b5bd7d318e349ccb7b4544e8353eac0` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:58:58.402660 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5bd7d318e349ccb7b4544e8353eac0)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:58:58.404592 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:58:58.412172 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:58:58.478233 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5bd7d318e349ccb7b4544e8353eac0', 'request_id': 'req-fe563358-e5b4-49aa-8237-ad22361c975c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:58.487396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5bd7d318e349ccb7b4544e8353eac0', 'request_id': 'req-fe563358-e5b4-49aa-8237-ad22361c975c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:58.494598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5bd7d318e349ccb7b4544e8353eac0', 'request_id': 'req-fe563358-e5b4-49aa-8237-ad22361c975c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:58.514445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] The token cache is being invalidate because project 9b5bd7d318e349ccb7b4544e8353eac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:58:58.514729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:58.515396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fe563358-e5b4-49aa-8237-ad22361c975c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:58:58.517171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1250/2488] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:58:58 2022] DELETE /identity/v3/projects/9b5bd7d318e349ccb7b4544e8353eac0 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:05.453272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:05.454784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:05.512802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a716d-39c6-4490-aef8-26b69ee6e858 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:05.560670 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80288429-cada-4f2a-9118-3f0dbd5da85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:05.561344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80288429-cada-4f2a-9118-3f0dbd5da85d service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:05.561656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80288429-cada-4f2a-9118-3f0dbd5da85d service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:05.561933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80288429-cada-4f2a-9118-3f0dbd5da85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:05.563728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-80288429-cada-4f2a-9118-3f0dbd5da85d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:05.630287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80288429-cada-4f2a-9118-3f0dbd5da85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:05.631712 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80288429-cada-4f2a-9118-3f0dbd5da85d service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:05.641214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1239/2489] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:06.649947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf258e6-ac36-4f94-ab2a-c62f807ae1bd None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:06.650458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf258e6-ac36-4f94-ab2a-c62f807ae1bd None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:06.650685 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf258e6-ac36-4f94-ab2a-c62f807ae1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:06.669635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-0cf258e6-ac36-4f94-ab2a-c62f807ae1bd None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:06.670890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0cf258e6-ac36-4f94-ab2a-c62f807ae1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:06.730620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1251/2490] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 1 19:59:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:06.743488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80288429-cada-4f2a-9118-3f0dbd5da85d service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:06.745083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-80288429-cada-4f2a-9118-3f0dbd5da85d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:06.792146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:06.817061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80288429-cada-4f2a-9118-3f0dbd5da85d service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:06.822289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726ca47-9032-403a-841d-e87d0dffaabd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:06.822933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726ca47-9032-403a-841d-e87d0dffaabd service swift] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:06.823214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726ca47-9032-403a-841d-e87d0dffaabd service swift] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:06.823491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726ca47-9032-403a-841d-e87d0dffaabd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:06.831667 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726ca47-9032-403a-841d-e87d0dffaabd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:06.833270 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726ca47-9032-403a-841d-e87d0dffaabd service swift] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:06.841620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1240/2491] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 19:59:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 19:59:09.112163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf258e6-ac36-4f94-ab2a-c62f807ae1bd None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:09.112701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0cf258e6-ac36-4f94-ab2a-c62f807ae1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:09.126620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf258e6-ac36-4f94-ab2a-c62f807ae1bd None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:09.131733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:09.132641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:09.132936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:09.133211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] PATH_INFO: `/v3/users/3198a63f6d1a4cdd88d76189658e79c0` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:09.134612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3198a63f6d1a4cdd88d76189658e79c0)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:09.136535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:09.182515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3198a63f6d1a4cdd88d76189658e79c0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:09.189760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3198a63f6d1a4cdd88d76189658e79c0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:09.192460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3198a63f6d1a4cdd88d76189658e79c0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:09.210150 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3198a63f6d1a4cdd88d76189658e79c0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:09.210786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-6dfcc4c8-1721-454c-939d-4a2a6e1d1628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3198a63f6d1a4cdd88d76189658e79c0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:09.223739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1252/2492] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:09 2022] DELETE /identity/v3/users/3198a63f6d1a4cdd88d76189658e79c0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:09.409184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726ca47-9032-403a-841d-e87d0dffaabd service swift] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:09.423042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726ca47-9032-403a-841d-e87d0dffaabd service swift] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:09.488647 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:09.489116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:09.489426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:09.489741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] PATH_INFO: `/v3/projects/025035b3b0b348d7acddfe456dd6a109` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:09.490932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=025035b3b0b348d7acddfe456dd6a109)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:09.492869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:09.539561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '025035b3b0b348d7acddfe456dd6a109', 'request_id': 'req-60200bf4-d494-4725-a508-354124ee7055'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:09.546680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '025035b3b0b348d7acddfe456dd6a109', 'request_id': 'req-60200bf4-d494-4725-a508-354124ee7055'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:09.552980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '025035b3b0b348d7acddfe456dd6a109', 'request_id': 'req-60200bf4-d494-4725-a508-354124ee7055'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:09.564023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] The token cache is being invalidate because project 025035b3b0b348d7acddfe456dd6a109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:09.564313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:09.564962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60200bf4-d494-4725-a508-354124ee7055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:09.567170 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1241/2493] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:09 2022] DELETE /identity/v3/projects/025035b3b0b348d7acddfe456dd6a109 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:09.572068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbe5a5-9598-434a-a66c-a9f6243579a7 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:09.572353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbe5a5-9598-434a-a66c-a9f6243579a7 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:09.572627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbe5a5-9598-434a-a66c-a9f6243579a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:09.591358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-eabbe5a5-9598-434a-a66c-a9f6243579a7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:09.592538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-eabbe5a5-9598-434a-a66c-a9f6243579a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:09.598249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:09.667689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1253/2494] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:09.672727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526062d-b3e9-4de5-937f-1ebf9f8ae78d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:09.673012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526062d-b3e9-4de5-937f-1ebf9f8ae78d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:09.673294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526062d-b3e9-4de5-937f-1ebf9f8ae78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:09.697334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-8526062d-b3e9-4de5-937f-1ebf9f8ae78d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:09.698625 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8526062d-b3e9-4de5-937f-1ebf9f8ae78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:09.706774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1242/2495] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:09.711099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbe5a5-9598-434a-a66c-a9f6243579a7 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:09.720671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbe5a5-9598-434a-a66c-a9f6243579a7 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:09.725716 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a070192-3d65-438b-843f-03628f7fea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:09.726368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a070192-3d65-438b-843f-03628f7fea5c admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:09.726651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a070192-3d65-438b-843f-03628f7fea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:09.726924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a070192-3d65-438b-843f-03628f7fea5c admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:09.727643 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a070192-3d65-438b-843f-03628f7fea5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:59:09.727963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a070192-3d65-438b-843f-03628f7fea5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:09.729562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a070192-3d65-438b-843f-03628f7fea5c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:09.742065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1254/2496] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:59:09.746237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526062d-b3e9-4de5-937f-1ebf9f8ae78d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:09.757723 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526062d-b3e9-4de5-937f-1ebf9f8ae78d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:09.762531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4037d9f-9934-4bae-babf-2bf19ea1ad40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:09.763189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4037d9f-9934-4bae-babf-2bf19ea1ad40 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:09.763470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4037d9f-9934-4bae-babf-2bf19ea1ad40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:09.763750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4037d9f-9934-4bae-babf-2bf19ea1ad40 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:09.764530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4037d9f-9934-4bae-babf-2bf19ea1ad40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:09.766491 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4037d9f-9934-4bae-babf-2bf19ea1ad40 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:09.805492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1243/2497] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:09 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:09.806612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a070192-3d65-438b-843f-03628f7fea5c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:09.817095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a070192-3d65-438b-843f-03628f7fea5c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:09.871836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff102a5-02c3-4188-9082-e0bb168a1552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:09.872491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff102a5-02c3-4188-9082-e0bb168a1552 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:09.872768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff102a5-02c3-4188-9082-e0bb168a1552 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:09.873038 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff102a5-02c3-4188-9082-e0bb168a1552 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:09.873744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff102a5-02c3-4188-9082-e0bb168a1552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:09.875656 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff102a5-02c3-4188-9082-e0bb168a1552 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:09.896850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1255/2498] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:09 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:09.900959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4037d9f-9934-4bae-babf-2bf19ea1ad40 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:09.911377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4037d9f-9934-4bae-babf-2bf19ea1ad40 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:09.916022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170d898-7927-4926-9b33-505bfb719ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:09.916671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170d898-7927-4926-9b33-505bfb719ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:09.916948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170d898-7927-4926-9b33-505bfb719ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:09.917218 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170d898-7927-4926-9b33-505bfb719ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:09.917956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170d898-7927-4926-9b33-505bfb719ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:09.919535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170d898-7927-4926-9b33-505bfb719ed1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:09.929804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1244/2499] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:09.933693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff102a5-02c3-4188-9082-e0bb168a1552 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:09.949184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff102a5-02c3-4188-9082-e0bb168a1552 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:09.954103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19d01f-50e7-43c9-bec6-373f1d3f59d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:09.954761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19d01f-50e7-43c9-bec6-373f1d3f59d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:09.955034 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19d01f-50e7-43c9-bec6-373f1d3f59d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:09.955326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19d01f-50e7-43c9-bec6-373f1d3f59d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:09.956042 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19d01f-50e7-43c9-bec6-373f1d3f59d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:09.958020 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19d01f-50e7-43c9-bec6-373f1d3f59d5 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:09.969308 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1256/2500] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:09.974315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170d898-7927-4926-9b33-505bfb719ed1 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:09.984484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170d898-7927-4926-9b33-505bfb719ed1 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:09.989883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8337d-ffaf-4306-83b5-6c8f3d2f6b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:09.990759 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8337d-ffaf-4306-83b5-6c8f3d2f6b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:09.991122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8337d-ffaf-4306-83b5-6c8f3d2f6b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:09.991403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8337d-ffaf-4306-83b5-6c8f3d2f6b2d admin admin] PATH_INFO: `/v3/projects/06638794c7444edcbc9497db49af27e2/users/9f5260158d0c4ed2a43d1a73817bf69c/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:10.003955 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:10.006943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8337d-ffaf-4306-83b5-6c8f3d2f6b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=06638794c7444edcbc9497db49af27e2, user_id=9f5260158d0c4ed2a43d1a73817bf69c, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:10.010586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8337d-ffaf-4306-83b5-6c8f3d2f6b2d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:10.024769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1245/2501] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:09 2022] PUT /identity/v3/projects/06638794c7444edcbc9497db49af27e2/users/9f5260158d0c4ed2a43d1a73817bf69c/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:10.035166 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb370292-2803-428f-be16-1658f08ac9f8 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:10.035478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb370292-2803-428f-be16-1658f08ac9f8 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:10.035748 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb370292-2803-428f-be16-1658f08ac9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:10.061354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-bb370292-2803-428f-be16-1658f08ac9f8 None None] MFA Rules not processed for user `9f5260158d0c4ed2a43d1a73817bf69c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:10.062580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-bb370292-2803-428f-be16-1658f08ac9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:10.126667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1257/2502] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:10.137794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8337d-ffaf-4306-83b5-6c8f3d2f6b2d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:10.139425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-eff8337d-ffaf-4306-83b5-6c8f3d2f6b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:10.213548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8337d-ffaf-4306-83b5-6c8f3d2f6b2d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:10.219069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa5fbc-a308-4bcb-891e-3a6c2dfc3435 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:10.219703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa5fbc-a308-4bcb-891e-3a6c2dfc3435 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:10.219982 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa5fbc-a308-4bcb-891e-3a6c2dfc3435 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:10.220250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa5fbc-a308-4bcb-891e-3a6c2dfc3435 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:10.228824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa5fbc-a308-4bcb-891e-3a6c2dfc3435 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:10.230418 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa5fbc-a308-4bcb-891e-3a6c2dfc3435 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:10.240076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1246/2503] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:17.497801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb370292-2803-428f-be16-1658f08ac9f8 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:17.506625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:17.562362 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb370292-2803-428f-be16-1658f08ac9f8 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:17.566822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:17.568181 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:17.568460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:17.568730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] PATH_INFO: `/v3/users/9f5260158d0c4ed2a43d1a73817bf69c` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:17.569928 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f5260158d0c4ed2a43d1a73817bf69c)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:17.571833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:17.606515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa5fbc-a308-4bcb-891e-3a6c2dfc3435 service nova] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:17.607997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e2fa5fbc-a308-4bcb-891e-3a6c2dfc3435 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:17.618483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f5260158d0c4ed2a43d1a73817bf69c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:17.626669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f5260158d0c4ed2a43d1a73817bf69c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:17.627348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f5260158d0c4ed2a43d1a73817bf69c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:17.639734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f5260158d0c4ed2a43d1a73817bf69c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:17.640326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-71a6578b-0f70-4a44-af2e-e61037705c5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f5260158d0c4ed2a43d1a73817bf69c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:17.653552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1258/2504] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:17 2022] DELETE /identity/v3/users/9f5260158d0c4ed2a43d1a73817bf69c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:17.657820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef26725-e71e-47a9-aa75-368a65f66b37 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:17.658140 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef26725-e71e-47a9-aa75-368a65f66b37 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:17.658417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef26725-e71e-47a9-aa75-368a65f66b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:17.669985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa5fbc-a308-4bcb-891e-3a6c2dfc3435 service nova] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:17.677396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5ef26725-e71e-47a9-aa75-368a65f66b37 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:17.678644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5ef26725-e71e-47a9-aa75-368a65f66b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:17.717985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e821c6-569c-498c-8e98-5abcd2e4a7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:17.718619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e821c6-569c-498c-8e98-5abcd2e4a7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:17.718892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e821c6-569c-498c-8e98-5abcd2e4a7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:17.719161 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e821c6-569c-498c-8e98-5abcd2e4a7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:17.720914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-89e821c6-569c-498c-8e98-5abcd2e4a7d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:17.739437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1259/2505] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:59:17.749959 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef26725-e71e-47a9-aa75-368a65f66b37 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:17.751328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5ef26725-e71e-47a9-aa75-368a65f66b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:17.763192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef26725-e71e-47a9-aa75-368a65f66b37 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:17.767173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9795c-07a0-474d-a77a-7facaa0eddf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:17.767809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9795c-07a0-474d-a77a-7facaa0eddf8 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:17.768089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9795c-07a0-474d-a77a-7facaa0eddf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:17.768367 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9795c-07a0-474d-a77a-7facaa0eddf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:17.768500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:17.778906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9795c-07a0-474d-a77a-7facaa0eddf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:17.780342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9795c-07a0-474d-a77a-7facaa0eddf8 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:17.784586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e821c6-569c-498c-8e98-5abcd2e4a7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:17.787099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e821c6-569c-498c-8e98-5abcd2e4a7d6 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:17.790762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1260/2506] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:59:17.796391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1247/2507] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:59:18.249897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9795c-07a0-474d-a77a-7facaa0eddf8 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:18.251418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-74a9795c-07a0-474d-a77a-7facaa0eddf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:18.280006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9795c-07a0-474d-a77a-7facaa0eddf8 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:18.285066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:18.286672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:18.287111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:18.287352 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] PATH_INFO: `/v3/projects/06638794c7444edcbc9497db49af27e2` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:18.288671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06638794c7444edcbc9497db49af27e2)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:18.290743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:18.299139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:18.343644 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06638794c7444edcbc9497db49af27e2', 'request_id': 'req-cf368d06-53c0-4851-a531-304fde5fe723'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:18.352631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06638794c7444edcbc9497db49af27e2', 'request_id': 'req-cf368d06-53c0-4851-a531-304fde5fe723'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:18.359205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06638794c7444edcbc9497db49af27e2', 'request_id': 'req-cf368d06-53c0-4851-a531-304fde5fe723'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:18.375694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] The token cache is being invalidate because project 06638794c7444edcbc9497db49af27e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:18.375980 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:18.376625 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cf368d06-53c0-4851-a531-304fde5fe723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:18.378403 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1261/2508] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:18 2022] DELETE /identity/v3/projects/06638794c7444edcbc9497db49af27e2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:18.383112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff661f-30be-4860-8efb-a4244ac613a7 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:18.383422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff661f-30be-4860-8efb-a4244ac613a7 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:18.383697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff661f-30be-4860-8efb-a4244ac613a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:18.403103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-edff661f-30be-4860-8efb-a4244ac613a7 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:18.404329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-edff661f-30be-4860-8efb-a4244ac613a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:18.466107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1248/2509] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:59:18.475062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702689df-83f2-46e1-8d99-0a5dc50b383c None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:18.475351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702689df-83f2-46e1-8d99-0a5dc50b383c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:18.475627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702689df-83f2-46e1-8d99-0a5dc50b383c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:18.496791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-702689df-83f2-46e1-8d99-0a5dc50b383c None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:18.497945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-702689df-83f2-46e1-8d99-0a5dc50b383c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:18.506901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1262/2510] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:59:18.511116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff661f-30be-4860-8efb-a4244ac613a7 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:18.522027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff661f-30be-4860-8efb-a4244ac613a7 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:18.528060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31aef29-9706-4dfb-9238-2362c7eccb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:18.528721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aef29-9706-4dfb-9238-2362c7eccb27 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:18.529005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aef29-9706-4dfb-9238-2362c7eccb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:18.529281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aef29-9706-4dfb-9238-2362c7eccb27 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:18.530048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31aef29-9706-4dfb-9238-2362c7eccb27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:59:18.530359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31aef29-9706-4dfb-9238-2362c7eccb27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:18.532030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31aef29-9706-4dfb-9238-2362c7eccb27 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:18.544630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1249/2511] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:59:18.549178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702689df-83f2-46e1-8d99-0a5dc50b383c None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:18.559741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702689df-83f2-46e1-8d99-0a5dc50b383c None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:18.564398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eecb06-8b58-4a0d-9d14-14410c38b09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:18.565044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eecb06-8b58-4a0d-9d14-14410c38b09d admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:18.565320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eecb06-8b58-4a0d-9d14-14410c38b09d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:18.565599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eecb06-8b58-4a0d-9d14-14410c38b09d admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:18.566371 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eecb06-8b58-4a0d-9d14-14410c38b09d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:18.569273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eecb06-8b58-4a0d-9d14-14410c38b09d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:18.598119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1263/2512] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:18 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:18.599852 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31aef29-9706-4dfb-9238-2362c7eccb27 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:18.610674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31aef29-9706-4dfb-9238-2362c7eccb27 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:18.668653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fce61-59a5-483d-af97-a47680b8b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:18.669340 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fce61-59a5-483d-af97-a47680b8b9de admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:18.669652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fce61-59a5-483d-af97-a47680b8b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:18.669932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fce61-59a5-483d-af97-a47680b8b9de admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:18.670622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fce61-59a5-483d-af97-a47680b8b9de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:18.672559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fce61-59a5-483d-af97-a47680b8b9de admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:18.697253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1250/2513] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:18 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:59:18.698514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eecb06-8b58-4a0d-9d14-14410c38b09d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:18.709273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eecb06-8b58-4a0d-9d14-14410c38b09d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:18.714316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57bc7e-d951-4ef7-a7b2-62139a7f7a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:18.715290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57bc7e-d951-4ef7-a7b2-62139a7f7a3b admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:18.715569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57bc7e-d951-4ef7-a7b2-62139a7f7a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:18.715842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57bc7e-d951-4ef7-a7b2-62139a7f7a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:18.716597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57bc7e-d951-4ef7-a7b2-62139a7f7a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:18.718237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57bc7e-d951-4ef7-a7b2-62139a7f7a3b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:18.729924 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1264/2514] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:18.734276 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fce61-59a5-483d-af97-a47680b8b9de admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:18.747334 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fce61-59a5-483d-af97-a47680b8b9de admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:18.752937 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63364b80-6c78-4abc-9475-8ef8fe48c75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:18.753635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364b80-6c78-4abc-9475-8ef8fe48c75c admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:18.753924 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364b80-6c78-4abc-9475-8ef8fe48c75c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:18.754203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364b80-6c78-4abc-9475-8ef8fe48c75c admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:18.754945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63364b80-6c78-4abc-9475-8ef8fe48c75c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:18.756547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63364b80-6c78-4abc-9475-8ef8fe48c75c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:18.768918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1251/2515] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:59:18.774085 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57bc7e-d951-4ef7-a7b2-62139a7f7a3b admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:18.805094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57bc7e-d951-4ef7-a7b2-62139a7f7a3b admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:18.810741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43180596-a23c-44c4-bb7f-c2b803d4305f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:18.811862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43180596-a23c-44c4-bb7f-c2b803d4305f admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:18.812223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43180596-a23c-44c4-bb7f-c2b803d4305f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:18.812705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43180596-a23c-44c4-bb7f-c2b803d4305f admin admin] PATH_INFO: `/v3/projects/ee3f6fe4f3ef4779961793f4cd85f95c/users/363c14e1f1d44a23acc06108f85b94e7/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:18.836245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43180596-a23c-44c4-bb7f-c2b803d4305f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3f6fe4f3ef4779961793f4cd85f95c, user_id=363c14e1f1d44a23acc06108f85b94e7, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:18.839051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43180596-a23c-44c4-bb7f-c2b803d4305f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:18.856237 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1265/2516] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:18 2022] PUT /identity/v3/projects/ee3f6fe4f3ef4779961793f4cd85f95c/users/363c14e1f1d44a23acc06108f85b94e7/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:18.870155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a61620-e644-45c9-a71b-44af7abf3456 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:18.870565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a61620-e644-45c9-a71b-44af7abf3456 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:18.871258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a61620-e644-45c9-a71b-44af7abf3456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:18.909007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-60a61620-e644-45c9-a71b-44af7abf3456 None None] MFA Rules not processed for user `363c14e1f1d44a23acc06108f85b94e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:18.911824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-60a61620-e644-45c9-a71b-44af7abf3456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:18.981288 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:19.010978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1252/2517] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:19.014707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43180596-a23c-44c4-bb7f-c2b803d4305f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:19.106173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43180596-a23c-44c4-bb7f-c2b803d4305f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:19.111084 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305d255-fb26-4fb0-abf2-2d7784dd012c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:19.111753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305d255-fb26-4fb0-abf2-2d7784dd012c admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:19.112036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305d255-fb26-4fb0-abf2-2d7784dd012c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:19.112668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305d255-fb26-4fb0-abf2-2d7784dd012c admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:19.113013 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305d255-fb26-4fb0-abf2-2d7784dd012c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:19.115265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305d255-fb26-4fb0-abf2-2d7784dd012c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:19.149398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1266/2518] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:19 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:59:19.153431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a61620-e644-45c9-a71b-44af7abf3456 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:19.167353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a61620-e644-45c9-a71b-44af7abf3456 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:19.229810 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2945c0-0ecc-49ac-9cff-d2749c292fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:19.230513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2945c0-0ecc-49ac-9cff-d2749c292fbd admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:19.230823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2945c0-0ecc-49ac-9cff-d2749c292fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:19.231096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2945c0-0ecc-49ac-9cff-d2749c292fbd admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:19.231783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2945c0-0ecc-49ac-9cff-d2749c292fbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:19.233721 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2945c0-0ecc-49ac-9cff-d2749c292fbd admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:19.254820 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1253/2519] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:19 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:19.259002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305d255-fb26-4fb0-abf2-2d7784dd012c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:19.269774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305d255-fb26-4fb0-abf2-2d7784dd012c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:19.275321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2f696-d521-48d3-9e22-edf856fc827f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:19.276100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2f696-d521-48d3-9e22-edf856fc827f admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:19.276451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2f696-d521-48d3-9e22-edf856fc827f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:19.276793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2f696-d521-48d3-9e22-edf856fc827f admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:19.277611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2f696-d521-48d3-9e22-edf856fc827f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:19.279286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2f696-d521-48d3-9e22-edf856fc827f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:19.291076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1267/2520] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:19.295510 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2945c0-0ecc-49ac-9cff-d2749c292fbd admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:19.306290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2945c0-0ecc-49ac-9cff-d2749c292fbd admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:19.310948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d2107-c60f-4fce-909b-1c54aa16cb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:19.311831 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d2107-c60f-4fce-909b-1c54aa16cb06 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:19.312109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d2107-c60f-4fce-909b-1c54aa16cb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:19.312383 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d2107-c60f-4fce-909b-1c54aa16cb06 admin admin] PATH_INFO: `/v3/projects/968ecece1fac4e9e8285a8e6342b0c35/users/d1b9c926156044ec886522aeea8ccee0/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:19.328534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d2107-c60f-4fce-909b-1c54aa16cb06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=968ecece1fac4e9e8285a8e6342b0c35, user_id=d1b9c926156044ec886522aeea8ccee0, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:19.331259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d2107-c60f-4fce-909b-1c54aa16cb06 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:19.344792 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1254/2521] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:19 2022] PUT /identity/v3/projects/968ecece1fac4e9e8285a8e6342b0c35/users/d1b9c926156044ec886522aeea8ccee0/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:19.348961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2f696-d521-48d3-9e22-edf856fc827f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:19.359847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2f696-d521-48d3-9e22-edf856fc827f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:19.365239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:19.427805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3196a6-fc4b-48f7-847b-7b2597579b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:19.428600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3196a6-fc4b-48f7-847b-7b2597579b56 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:19.428951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3196a6-fc4b-48f7-847b-7b2597579b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:19.429302 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3196a6-fc4b-48f7-847b-7b2597579b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:19.430096 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3196a6-fc4b-48f7-847b-7b2597579b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:19.431716 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3196a6-fc4b-48f7-847b-7b2597579b56 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:19.443000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1268/2522] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:19.447482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d2107-c60f-4fce-909b-1c54aa16cb06 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:19.458014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d2107-c60f-4fce-909b-1c54aa16cb06 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:19.463017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9bb9b-899f-4e4d-bacc-3b8cf7682474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:19.464032 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9bb9b-899f-4e4d-bacc-3b8cf7682474 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:19.464316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9bb9b-899f-4e4d-bacc-3b8cf7682474 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:19.464715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9bb9b-899f-4e4d-bacc-3b8cf7682474 admin admin] PATH_INFO: `/v3/projects/968ecece1fac4e9e8285a8e6342b0c35/users/d1b9c926156044ec886522aeea8ccee0/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:19.466760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9bb9b-899f-4e4d-bacc-3b8cf7682474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=968ecece1fac4e9e8285a8e6342b0c35, user_id=d1b9c926156044ec886522aeea8ccee0, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:19.469424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9bb9b-899f-4e4d-bacc-3b8cf7682474 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:19.480461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1255/2523] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:19 2022] PUT /identity/v3/projects/968ecece1fac4e9e8285a8e6342b0c35/users/d1b9c926156044ec886522aeea8ccee0/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:59:19.491775 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7df2e-d7a4-4b96-a705-7ea0bd9e2b53 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:19.492069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7df2e-d7a4-4b96-a705-7ea0bd9e2b53 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:19.492346 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7df2e-d7a4-4b96-a705-7ea0bd9e2b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:19.523296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-04e7df2e-d7a4-4b96-a705-7ea0bd9e2b53 None None] MFA Rules not processed for user `d1b9c926156044ec886522aeea8ccee0`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:19.524439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-04e7df2e-d7a4-4b96-a705-7ea0bd9e2b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:19.596692 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1269/2524] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:19.608269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9bb9b-899f-4e4d-bacc-3b8cf7682474 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:19.610003 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-71d9bb9b-899f-4e4d-bacc-3b8cf7682474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:19.672104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9bb9b-899f-4e4d-bacc-3b8cf7682474 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:19.676023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ae798-8fc6-445f-bd3a-2635b3505999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:19.676669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ae798-8fc6-445f-bd3a-2635b3505999 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:19.676952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ae798-8fc6-445f-bd3a-2635b3505999 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:19.677229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ae798-8fc6-445f-bd3a-2635b3505999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:19.687234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ae798-8fc6-445f-bd3a-2635b3505999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:19.688685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ae798-8fc6-445f-bd3a-2635b3505999 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:19.700337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1256/2525] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:20.094245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7df2e-d7a4-4b96-a705-7ea0bd9e2b53 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:20.160829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7df2e-d7a4-4b96-a705-7ea0bd9e2b53 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:20.166372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:20.167254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:20.167530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:20.167798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] PATH_INFO: `/v3/users/363c14e1f1d44a23acc06108f85b94e7` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:20.168973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=363c14e1f1d44a23acc06108f85b94e7)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:20.170925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:20.205858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:20.221483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '363c14e1f1d44a23acc06108f85b94e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:20.231709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '363c14e1f1d44a23acc06108f85b94e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:20.232515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '363c14e1f1d44a23acc06108f85b94e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:20.245139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '363c14e1f1d44a23acc06108f85b94e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:20.245840 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '363c14e1f1d44a23acc06108f85b94e7'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:20.258482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1270/2526] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:20 2022] DELETE /identity/v3/users/363c14e1f1d44a23acc06108f85b94e7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:20.263034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd89678-ee75-423e-bc06-a72813ae8f0b None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:20.263316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd89678-ee75-423e-bc06-a72813ae8f0b None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:20.263590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd89678-ee75-423e-bc06-a72813ae8f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:20.284744 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-afd89678-ee75-423e-bc06-a72813ae8f0b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:20.286859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-afd89678-ee75-423e-bc06-a72813ae8f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:20.322655 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:20.359009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1257/2527] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:20.369074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:20.370636 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:20.443895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be3699-ec90-47d5-92fd-05d65281da1c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:20.447565 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2af13-f440-41a8-8ee5-ae6e52121993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:20.448205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2af13-f440-41a8-8ee5-ae6e52121993 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:20.448485 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2af13-f440-41a8-8ee5-ae6e52121993 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:20.448754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2af13-f440-41a8-8ee5-ae6e52121993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:20.459557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2af13-f440-41a8-8ee5-ae6e52121993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:20.461002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2af13-f440-41a8-8ee5-ae6e52121993 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:20.475961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1271/2528] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:20.760221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd89678-ee75-423e-bc06-a72813ae8f0b None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:20.762633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-afd89678-ee75-423e-bc06-a72813ae8f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:20.777678 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd89678-ee75-423e-bc06-a72813ae8f0b None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:20.783200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:20.784315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:20.784435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:20.784834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] PATH_INFO: `/v3/projects/ee3f6fe4f3ef4779961793f4cd85f95c` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:20.786441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee3f6fe4f3ef4779961793f4cd85f95c)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:20.788738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:20.841426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee3f6fe4f3ef4779961793f4cd85f95c', 'request_id': 'req-ee391fd0-6406-4410-a36b-76b4401af022'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:20.849750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee3f6fe4f3ef4779961793f4cd85f95c', 'request_id': 'req-ee391fd0-6406-4410-a36b-76b4401af022'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:20.856085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee3f6fe4f3ef4779961793f4cd85f95c', 'request_id': 'req-ee391fd0-6406-4410-a36b-76b4401af022'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:20.868984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] The token cache is being invalidate because project ee3f6fe4f3ef4779961793f4cd85f95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:20.869290 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:20.869984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:20.871903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1258/2529] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:20 2022] DELETE /identity/v3/projects/ee3f6fe4f3ef4779961793f4cd85f95c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:20.875566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2af13-f440-41a8-8ee5-ae6e52121993 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:20.886406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2af13-f440-41a8-8ee5-ae6e52121993 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:20.947682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:20.948584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:20.948941 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:20.949248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] PATH_INFO: `/v3/users/d1b9c926156044ec886522aeea8ccee0` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:20.950599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b9c926156044ec886522aeea8ccee0)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:20.952712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:21.002124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b9c926156044ec886522aeea8ccee0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:21.007529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:21.012239 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b9c926156044ec886522aeea8ccee0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:21.013664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b9c926156044ec886522aeea8ccee0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:21.027889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b9c926156044ec886522aeea8ccee0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:21.028635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-a26c2485-22d9-4962-a1f6-c013f586d9e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b9c926156044ec886522aeea8ccee0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:21.048053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1272/2530] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:20 2022] DELETE /identity/v3/users/d1b9c926156044ec886522aeea8ccee0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:21.367330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:21.368961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:21.385043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee391fd0-6406-4410-a36b-76b4401af022 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:21.453035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:21.453942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:21.454221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:21.454496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] PATH_INFO: `/v3/projects/968ecece1fac4e9e8285a8e6342b0c35` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:21.455787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=968ecece1fac4e9e8285a8e6342b0c35)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:21.458228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:21.528834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '968ecece1fac4e9e8285a8e6342b0c35', 'request_id': 'req-12761384-6dec-4097-acf9-3ecfb7f77530'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:21.541741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '968ecece1fac4e9e8285a8e6342b0c35', 'request_id': 'req-12761384-6dec-4097-acf9-3ecfb7f77530'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:21.556880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:21.562081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '968ecece1fac4e9e8285a8e6342b0c35', 'request_id': 'req-12761384-6dec-4097-acf9-3ecfb7f77530'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:21.587445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] The token cache is being invalidate because project 968ecece1fac4e9e8285a8e6342b0c35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:21.587618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:21.588454 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-12761384-6dec-4097-acf9-3ecfb7f77530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:21.590357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1259/2531] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:21 2022] DELETE /identity/v3/projects/968ecece1fac4e9e8285a8e6342b0c35 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:21.595086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f8041-13ca-4f0c-bfef-607d9d163d68 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:21.595406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f8041-13ca-4f0c-bfef-607d9d163d68 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:21.595682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f8041-13ca-4f0c-bfef-607d9d163d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:21.615431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-276f8041-13ca-4f0c-bfef-607d9d163d68 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:21.616620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-276f8041-13ca-4f0c-bfef-607d9d163d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:21.682021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1273/2532] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:21.691105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97983b3b-16f9-4a29-aa14-3c11f773ed54 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:21.691358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97983b3b-16f9-4a29-aa14-3c11f773ed54 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:21.691887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97983b3b-16f9-4a29-aa14-3c11f773ed54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:21.711729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-97983b3b-16f9-4a29-aa14-3c11f773ed54 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:21.712914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-97983b3b-16f9-4a29-aa14-3c11f773ed54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:21.720691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1260/2533] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:21.726038 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f8041-13ca-4f0c-bfef-607d9d163d68 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:21.735961 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f8041-13ca-4f0c-bfef-607d9d163d68 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:21.740987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8ae2e-d5be-4397-b13c-1a8c2e123cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:21.741679 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8ae2e-d5be-4397-b13c-1a8c2e123cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:21.741963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8ae2e-d5be-4397-b13c-1a8c2e123cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:21.742236 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8ae2e-d5be-4397-b13c-1a8c2e123cf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:21.742956 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8ae2e-d5be-4397-b13c-1a8c2e123cf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:59:21.743260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8ae2e-d5be-4397-b13c-1a8c2e123cf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:21.744870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8ae2e-d5be-4397-b13c-1a8c2e123cf4 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:21.756560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1274/2534] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:59:21.760765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97983b3b-16f9-4a29-aa14-3c11f773ed54 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:21.771081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97983b3b-16f9-4a29-aa14-3c11f773ed54 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:21.775769 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487650b8-d34b-4336-977e-baa53e67c7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:21.776404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487650b8-d34b-4336-977e-baa53e67c7dd admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:21.776694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487650b8-d34b-4336-977e-baa53e67c7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:21.776958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487650b8-d34b-4336-977e-baa53e67c7dd admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:21.777694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487650b8-d34b-4336-977e-baa53e67c7dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:21.779592 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487650b8-d34b-4336-977e-baa53e67c7dd admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:21.804687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1261/2535] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:21 2022] POST /identity/v3/projects => generated 399 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:21.808850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8ae2e-d5be-4397-b13c-1a8c2e123cf4 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:21.819857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8ae2e-d5be-4397-b13c-1a8c2e123cf4 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:21.880213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2854cf0-99f0-4ca7-93c1-681222072e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:21.880955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2854cf0-99f0-4ca7-93c1-681222072e17 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:21.881238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2854cf0-99f0-4ca7-93c1-681222072e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:21.881537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2854cf0-99f0-4ca7-93c1-681222072e17 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:21.882271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2854cf0-99f0-4ca7-93c1-681222072e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:21.884430 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2854cf0-99f0-4ca7-93c1-681222072e17 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:21.906652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1275/2536] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:21 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:21.911537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487650b8-d34b-4336-977e-baa53e67c7dd admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:21.921499 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487650b8-d34b-4336-977e-baa53e67c7dd admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:21.926652 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff572c00-b420-4c10-83ef-2aee6501ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:21.927311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff572c00-b420-4c10-83ef-2aee6501ccf8 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:21.927609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff572c00-b420-4c10-83ef-2aee6501ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:21.927859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff572c00-b420-4c10-83ef-2aee6501ccf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:21.928583 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff572c00-b420-4c10-83ef-2aee6501ccf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:21.930206 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff572c00-b420-4c10-83ef-2aee6501ccf8 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:21.941478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1262/2537] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:21.946056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2854cf0-99f0-4ca7-93c1-681222072e17 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:21.958232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2854cf0-99f0-4ca7-93c1-681222072e17 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:21.962942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4d579-2220-4399-94a1-4c3348129e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:21.963610 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4d579-2220-4399-94a1-4c3348129e38 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:21.963889 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4d579-2220-4399-94a1-4c3348129e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:21.964163 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4d579-2220-4399-94a1-4c3348129e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:21.964910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4d579-2220-4399-94a1-4c3348129e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:21.966533 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4d579-2220-4399-94a1-4c3348129e38 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:21.978060 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1276/2538] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:21.982717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff572c00-b420-4c10-83ef-2aee6501ccf8 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:21.992983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff572c00-b420-4c10-83ef-2aee6501ccf8 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:21.997516 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df50a48-c9e6-4345-8938-423892ea9ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:21.998415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df50a48-c9e6-4345-8938-423892ea9ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:21.998696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df50a48-c9e6-4345-8938-423892ea9ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:21.998985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df50a48-c9e6-4345-8938-423892ea9ac0 admin admin] PATH_INFO: `/v3/projects/129db714af2042bd99bdc5d9f1d06f3b/users/52d68e3a9a9c4abf9f4f6198f319af9c/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.013644 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df50a48-c9e6-4345-8938-423892ea9ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129db714af2042bd99bdc5d9f1d06f3b, user_id=52d68e3a9a9c4abf9f4f6198f319af9c, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:22.016287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df50a48-c9e6-4345-8938-423892ea9ac0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:22.029650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1263/2539] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:21 2022] PUT /identity/v3/projects/129db714af2042bd99bdc5d9f1d06f3b/users/52d68e3a9a9c4abf9f4f6198f319af9c/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:59:22.040025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42538f-5d78-4102-9170-6b0fcb5a64f0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.040314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42538f-5d78-4102-9170-6b0fcb5a64f0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.040591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42538f-5d78-4102-9170-6b0fcb5a64f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.067056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6c42538f-5d78-4102-9170-6b0fcb5a64f0 None None] MFA Rules not processed for user `52d68e3a9a9c4abf9f4f6198f319af9c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:22.068780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6c42538f-5d78-4102-9170-6b0fcb5a64f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:22.092918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:22.134835 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1277/2540] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:22.139223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df50a48-c9e6-4345-8938-423892ea9ac0 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:22.200437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df50a48-c9e6-4345-8938-423892ea9ac0 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:22.205051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99749cad-5546-42e3-8f2b-7aba4a3c2cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:22.205730 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99749cad-5546-42e3-8f2b-7aba4a3c2cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.206018 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99749cad-5546-42e3-8f2b-7aba4a3c2cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.206298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99749cad-5546-42e3-8f2b-7aba4a3c2cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.206991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99749cad-5546-42e3-8f2b-7aba4a3c2cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:22.209640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99749cad-5546-42e3-8f2b-7aba4a3c2cf9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:22.234179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1264/2541] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:22 2022] POST /identity/v3/projects => generated 401 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:22.238301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42538f-5d78-4102-9170-6b0fcb5a64f0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:22.249016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42538f-5d78-4102-9170-6b0fcb5a64f0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:22.308786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37b36a-9739-4434-ba49-90bca54b3078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:22.309548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37b36a-9739-4434-ba49-90bca54b3078 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.309922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37b36a-9739-4434-ba49-90bca54b3078 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.310270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37b36a-9739-4434-ba49-90bca54b3078 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.311019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37b36a-9739-4434-ba49-90bca54b3078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:22.313016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37b36a-9739-4434-ba49-90bca54b3078 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:22.333871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1278/2542] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:22 2022] POST /identity/v3/users => generated 307 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:22.337617 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99749cad-5546-42e3-8f2b-7aba4a3c2cf9 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:22.347790 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99749cad-5546-42e3-8f2b-7aba4a3c2cf9 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:22.352398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b35dae-9d24-4d02-94ba-da074f28327c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:22.353046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b35dae-9d24-4d02-94ba-da074f28327c admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.353328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b35dae-9d24-4d02-94ba-da074f28327c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.353628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b35dae-9d24-4d02-94ba-da074f28327c admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.354348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b35dae-9d24-4d02-94ba-da074f28327c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:22.355933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b35dae-9d24-4d02-94ba-da074f28327c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:22.366784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1265/2543] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:22.370696 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37b36a-9739-4434-ba49-90bca54b3078 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:22.381730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37b36a-9739-4434-ba49-90bca54b3078 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:22.386423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3597a1-fc61-4bd6-9e0b-bc5090485e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:22.387244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3597a1-fc61-4bd6-9e0b-bc5090485e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.387524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3597a1-fc61-4bd6-9e0b-bc5090485e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.387800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3597a1-fc61-4bd6-9e0b-bc5090485e9e admin admin] PATH_INFO: `/v3/projects/5f8fea8dd4e143ec8f6acefca6d232dc/users/838049fe9d8a480191bdb905fd412777/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.402430 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3597a1-fc61-4bd6-9e0b-bc5090485e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8fea8dd4e143ec8f6acefca6d232dc, user_id=838049fe9d8a480191bdb905fd412777, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:22.405088 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3597a1-fc61-4bd6-9e0b-bc5090485e9e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:22.417829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1279/2544] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:22 2022] PUT /identity/v3/projects/5f8fea8dd4e143ec8f6acefca6d232dc/users/838049fe9d8a480191bdb905fd412777/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:59:22.421763 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b35dae-9d24-4d02-94ba-da074f28327c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:22.432230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b35dae-9d24-4d02-94ba-da074f28327c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:22.489455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84d723-8438-49d7-8c38-f04b846a5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:22.490133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84d723-8438-49d7-8c38-f04b846a5a13 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.490416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84d723-8438-49d7-8c38-f04b846a5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.490691 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84d723-8438-49d7-8c38-f04b846a5a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.491411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84d723-8438-49d7-8c38-f04b846a5a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:22.493011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84d723-8438-49d7-8c38-f04b846a5a13 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:22.495169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:22.505083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1266/2545] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:22.508790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3597a1-fc61-4bd6-9e0b-bc5090485e9e admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:22.519105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3597a1-fc61-4bd6-9e0b-bc5090485e9e admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:22.523938 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0c457-f57b-4dc0-84a4-81cc059155d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:22.524833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0c457-f57b-4dc0-84a4-81cc059155d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.525100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0c457-f57b-4dc0-84a4-81cc059155d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.525372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0c457-f57b-4dc0-84a4-81cc059155d7 admin admin] PATH_INFO: `/v3/projects/5f8fea8dd4e143ec8f6acefca6d232dc/users/838049fe9d8a480191bdb905fd412777/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.527339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0c457-f57b-4dc0-84a4-81cc059155d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8fea8dd4e143ec8f6acefca6d232dc, user_id=838049fe9d8a480191bdb905fd412777, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:22.530016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0c457-f57b-4dc0-84a4-81cc059155d7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:22.540168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1280/2546] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:22 2022] PUT /identity/v3/projects/5f8fea8dd4e143ec8f6acefca6d232dc/users/838049fe9d8a480191bdb905fd412777/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:59:22.550561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd2add-55d8-48bd-b0b6-cb1fc4d870af None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.551447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd2add-55d8-48bd-b0b6-cb1fc4d870af None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.551876 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd2add-55d8-48bd-b0b6-cb1fc4d870af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.578412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-8ebd2add-55d8-48bd-b0b6-cb1fc4d870af None None] MFA Rules not processed for user `838049fe9d8a480191bdb905fd412777`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:22.579803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-8ebd2add-55d8-48bd-b0b6-cb1fc4d870af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:22.655993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1267/2547] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:22 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:22.668283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0c457-f57b-4dc0-84a4-81cc059155d7 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:22.670907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-80a0c457-f57b-4dc0-84a4-81cc059155d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:22.742811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0c457-f57b-4dc0-84a4-81cc059155d7 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:22.746480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cad5d-6d42-49eb-a518-9b5148ad060f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:22.747127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cad5d-6d42-49eb-a518-9b5148ad060f service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.747409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cad5d-6d42-49eb-a518-9b5148ad060f service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.747683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337cad5d-6d42-49eb-a518-9b5148ad060f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.757069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cad5d-6d42-49eb-a518-9b5148ad060f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:22.758541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337cad5d-6d42-49eb-a518-9b5148ad060f service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:22.768079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1281/2548] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:22 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:22.855671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd2add-55d8-48bd-b0b6-cb1fc4d870af None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:22.857300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cad5d-6d42-49eb-a518-9b5148ad060f service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:22.858818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-337cad5d-6d42-49eb-a518-9b5148ad060f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:22.868115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd2add-55d8-48bd-b0b6-cb1fc4d870af None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:22.872103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a3d7b-429a-4e75-8f58-ac463c69e4e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:22.872741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a3d7b-429a-4e75-8f58-ac463c69e4e3 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.873184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a3d7b-429a-4e75-8f58-ac463c69e4e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.873533 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a3d7b-429a-4e75-8f58-ac463c69e4e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.882113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:22.920479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337cad5d-6d42-49eb-a518-9b5148ad060f service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:22.924191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2df66-ccc2-4191-9f48-a3808d623ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:22.924905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2df66-ccc2-4191-9f48-a3808d623ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:22.925257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2df66-ccc2-4191-9f48-a3808d623ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:22.925561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2df66-ccc2-4191-9f48-a3808d623ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:22.928241 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a5c2df66-ccc2-4191-9f48-a3808d623ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:22.934913 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a3d7b-429a-4e75-8f58-ac463c69e4e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:22.936360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a3d7b-429a-4e75-8f58-ac463c69e4e3 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:22.945500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1268/2549] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:23.001985 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2df66-ccc2-4191-9f48-a3808d623ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:23.003829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2df66-ccc2-4191-9f48-a3808d623ca7 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:23.015705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1282/2550] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:25.197865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbed79-dfb3-452b-be0b-b47b693da741 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:25.198696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbed79-dfb3-452b-be0b-b47b693da741 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:25.199133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbed79-dfb3-452b-be0b-b47b693da741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:25.236390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-92bbed79-dfb3-452b-be0b-b47b693da741 None None] MFA Rules not processed for user `8f2b9a6934d34f58a09a4cefb154371c`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:25.237959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-92bbed79-dfb3-452b-be0b-b47b693da741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:25.305667 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1269/2551] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:26.924493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2df66-ccc2-4191-9f48-a3808d623ca7 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:26.927247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a5c2df66-ccc2-4191-9f48-a3808d623ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:27.049225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2df66-ccc2-4191-9f48-a3808d623ca7 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:27.055208 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4d0c7-753d-426e-bb31-26cf99153d3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:27.055897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4d0c7-753d-426e-bb31-26cf99153d3a service placement] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:27.056167 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4d0c7-753d-426e-bb31-26cf99153d3a service placement] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:27.056442 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4d0c7-753d-426e-bb31-26cf99153d3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:27.058270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e9e4d0c7-753d-426e-bb31-26cf99153d3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:27.072265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4d0c7-753d-426e-bb31-26cf99153d3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:27.074182 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4d0c7-753d-426e-bb31-26cf99153d3a service placement] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:27.087858 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1283/2552] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:27.197540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbed79-dfb3-452b-be0b-b47b693da741 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:27.209487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbed79-dfb3-452b-be0b-b47b693da741 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:27.219978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:27.220877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:27.221243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:27.221745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] PATH_INFO: `/v3/users/52d68e3a9a9c4abf9f4f6198f319af9c` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:27.223461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d68e3a9a9c4abf9f4f6198f319af9c)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:27.225598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:27.250486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:27.282606 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d68e3a9a9c4abf9f4f6198f319af9c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:27.301617 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d68e3a9a9c4abf9f4f6198f319af9c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:27.310622 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d68e3a9a9c4abf9f4f6198f319af9c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:27.321591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d68e3a9a9c4abf9f4f6198f319af9c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:27.322730 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d68e3a9a9c4abf9f4f6198f319af9c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:27.329859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1270/2553] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:27 2022] DELETE /identity/v3/users/52d68e3a9a9c4abf9f4f6198f319af9c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:27.335094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093b867-8c4b-4bca-a722-36ba9f0acd44 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:27.335965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093b867-8c4b-4bca-a722-36ba9f0acd44 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:27.336411 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093b867-8c4b-4bca-a722-36ba9f0acd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:27.358842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-b093b867-8c4b-4bca-a722-36ba9f0acd44 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:27.360064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b093b867-8c4b-4bca-a722-36ba9f0acd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:27.423719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:27.432663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1284/2554] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:27.442668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:27.444158 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:27.531041 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba0e88-9269-417f-8a08-04d66bc77896 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:27.537407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4029d-bf13-46dd-90eb-ffa7323a2406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:27.538477 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4029d-bf13-46dd-90eb-ffa7323a2406 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:27.538760 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4029d-bf13-46dd-90eb-ffa7323a2406 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:27.539644 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4029d-bf13-46dd-90eb-ffa7323a2406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:27.555706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4029d-bf13-46dd-90eb-ffa7323a2406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:27.557395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4029d-bf13-46dd-90eb-ffa7323a2406 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:27.569780 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1271/2555] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 19:59:27.866015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093b867-8c4b-4bca-a722-36ba9f0acd44 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:27.867580 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b093b867-8c4b-4bca-a722-36ba9f0acd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:27.880604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093b867-8c4b-4bca-a722-36ba9f0acd44 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:27.885522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:27.886531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:27.886882 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:27.887215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] PATH_INFO: `/v3/projects/129db714af2042bd99bdc5d9f1d06f3b` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:27.888493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=129db714af2042bd99bdc5d9f1d06f3b)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:27.890582 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:27.940908 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129db714af2042bd99bdc5d9f1d06f3b', 'request_id': 'req-22a18d37-efd6-42ab-afc9-154b75734e2c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:27.952416 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129db714af2042bd99bdc5d9f1d06f3b', 'request_id': 'req-22a18d37-efd6-42ab-afc9-154b75734e2c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:27.960511 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129db714af2042bd99bdc5d9f1d06f3b', 'request_id': 'req-22a18d37-efd6-42ab-afc9-154b75734e2c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:27.971750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] The token cache is being invalidate because project 129db714af2042bd99bdc5d9f1d06f3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:27.972040 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:27.972782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:27.974365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1285/2556] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:27 2022] DELETE /identity/v3/projects/129db714af2042bd99bdc5d9f1d06f3b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:27.978433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4029d-bf13-46dd-90eb-ffa7323a2406 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:27.988880 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4029d-bf13-46dd-90eb-ffa7323a2406 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:28.046969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:28.047766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:28.048052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:28.048327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] PATH_INFO: `/v3/users/838049fe9d8a480191bdb905fd412777` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:28.049536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=838049fe9d8a480191bdb905fd412777)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:28.051561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:28.100271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:28.103534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '838049fe9d8a480191bdb905fd412777'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:28.119985 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '838049fe9d8a480191bdb905fd412777'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:28.128504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '838049fe9d8a480191bdb905fd412777'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:28.136430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '838049fe9d8a480191bdb905fd412777'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:28.137089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-5a6815ce-d0bd-4001-8aa5-d64f7cbbc8cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '838049fe9d8a480191bdb905fd412777'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:28.143442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1272/2557] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:27 2022] DELETE /identity/v3/users/838049fe9d8a480191bdb905fd412777 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:28.443691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:28.446189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:28.471270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a18d37-efd6-42ab-afc9-154b75734e2c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:28.554185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:28.555254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:28.555664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:28.556137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] PATH_INFO: `/v3/projects/5f8fea8dd4e143ec8f6acefca6d232dc` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:28.557782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f8fea8dd4e143ec8f6acefca6d232dc)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:28.559877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:28.616001 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f8fea8dd4e143ec8f6acefca6d232dc', 'request_id': 'req-1aa49448-defd-4579-9803-0b7b966fc450'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:28.625247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f8fea8dd4e143ec8f6acefca6d232dc', 'request_id': 'req-1aa49448-defd-4579-9803-0b7b966fc450'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:28.632920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f8fea8dd4e143ec8f6acefca6d232dc', 'request_id': 'req-1aa49448-defd-4579-9803-0b7b966fc450'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:28.646161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] The token cache is being invalidate because project 5f8fea8dd4e143ec8f6acefca6d232dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:28.646450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:28.647129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-1aa49448-defd-4579-9803-0b7b966fc450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:28.649200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1286/2558] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:28 2022] DELETE /identity/v3/projects/5f8fea8dd4e143ec8f6acefca6d232dc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:28.654194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1438446-e811-42b5-8092-2c2620f89390 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:28.654693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1438446-e811-42b5-8092-2c2620f89390 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:28.654693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1438446-e811-42b5-8092-2c2620f89390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:28.677327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d1438446-e811-42b5-8092-2c2620f89390 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:28.678878 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d1438446-e811-42b5-8092-2c2620f89390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:28.740297 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1273/2559] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:59:28.749414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d8079-18dc-47e2-9231-a853211e5896 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:28.749885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d8079-18dc-47e2-9231-a853211e5896 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:28.750385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d8079-18dc-47e2-9231-a853211e5896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:28.769776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-bf0d8079-18dc-47e2-9231-a853211e5896 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:28.771445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-bf0d8079-18dc-47e2-9231-a853211e5896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:28.779680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1287/2560] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:28.784179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1438446-e811-42b5-8092-2c2620f89390 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:28.795361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1438446-e811-42b5-8092-2c2620f89390 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:28.800077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38da18-e99b-457b-9749-af4184a1c413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:28.801615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38da18-e99b-457b-9749-af4184a1c413 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:28.801912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38da18-e99b-457b-9749-af4184a1c413 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:28.802193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38da18-e99b-457b-9749-af4184a1c413 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:28.802943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38da18-e99b-457b-9749-af4184a1c413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:59:28.803252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38da18-e99b-457b-9749-af4184a1c413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:28.804989 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38da18-e99b-457b-9749-af4184a1c413 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:28.818296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1274/2561] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:59:28.822883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d8079-18dc-47e2-9231-a853211e5896 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:28.836130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d8079-18dc-47e2-9231-a853211e5896 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:28.840891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206002-9ed7-4579-8036-abcea2aaca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:28.841632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206002-9ed7-4579-8036-abcea2aaca48 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:28.841918 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206002-9ed7-4579-8036-abcea2aaca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:28.842191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206002-9ed7-4579-8036-abcea2aaca48 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:28.842903 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6206002-9ed7-4579-8036-abcea2aaca48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:28.844847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6206002-9ed7-4579-8036-abcea2aaca48 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:28.849028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:28.880491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1288/2562] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:28 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:28.882033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38da18-e99b-457b-9749-af4184a1c413 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:28.892521 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38da18-e99b-457b-9749-af4184a1c413 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:28.954377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9644c-8b37-4644-85e0-5abd074046c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:28.955033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9644c-8b37-4644-85e0-5abd074046c4 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:28.955314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9644c-8b37-4644-85e0-5abd074046c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:28.955593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9644c-8b37-4644-85e0-5abd074046c4 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:28.956282 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9644c-8b37-4644-85e0-5abd074046c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:28.958326 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9644c-8b37-4644-85e0-5abd074046c4 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:28.980132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1275/2563] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:28 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:28.984331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206002-9ed7-4579-8036-abcea2aaca48 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:28.996829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206002-9ed7-4579-8036-abcea2aaca48 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:29.001542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1391062-eb57-4f98-84d5-2a4999cc3981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:29.002255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1391062-eb57-4f98-84d5-2a4999cc3981 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.002538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1391062-eb57-4f98-84d5-2a4999cc3981 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.002807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1391062-eb57-4f98-84d5-2a4999cc3981 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.003560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1391062-eb57-4f98-84d5-2a4999cc3981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:29.005536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1391062-eb57-4f98-84d5-2a4999cc3981 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:29.016118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1289/2564] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:29.021121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9644c-8b37-4644-85e0-5abd074046c4 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:29.032149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9644c-8b37-4644-85e0-5abd074046c4 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:29.037686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a36ec-3f08-4e32-8bcf-45c73786c895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:29.038352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a36ec-3f08-4e32-8bcf-45c73786c895 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.038637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a36ec-3f08-4e32-8bcf-45c73786c895 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.038914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a36ec-3f08-4e32-8bcf-45c73786c895 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.039768 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a36ec-3f08-4e32-8bcf-45c73786c895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:29.041891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a36ec-3f08-4e32-8bcf-45c73786c895 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:29.053115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1276/2565] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:29.057201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1391062-eb57-4f98-84d5-2a4999cc3981 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:29.067421 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1391062-eb57-4f98-84d5-2a4999cc3981 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:29.072223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3742ab0-82c3-4d5a-b76a-52f047470e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:29.073191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3742ab0-82c3-4d5a-b76a-52f047470e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.073540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3742ab0-82c3-4d5a-b76a-52f047470e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.073855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3742ab0-82c3-4d5a-b76a-52f047470e4b admin admin] PATH_INFO: `/v3/projects/a43d5410b59d4b17ac00aaefb4a2f1ff/users/7c31befc1f9f46cfa4f77a08db763421/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.089852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3742ab0-82c3-4d5a-b76a-52f047470e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a43d5410b59d4b17ac00aaefb4a2f1ff, user_id=7c31befc1f9f46cfa4f77a08db763421, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:29.092689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3742ab0-82c3-4d5a-b76a-52f047470e4b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:29.107673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1290/2566] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:29 2022] PUT /identity/v3/projects/a43d5410b59d4b17ac00aaefb4a2f1ff/users/7c31befc1f9f46cfa4f77a08db763421/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:29.131903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af0a10-958f-4848-8a07-d2b311fb4042 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.132275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af0a10-958f-4848-8a07-d2b311fb4042 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.132673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af0a10-958f-4848-8a07-d2b311fb4042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.164271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f5af0a10-958f-4848-8a07-d2b311fb4042 None None] MFA Rules not processed for user `7c31befc1f9f46cfa4f77a08db763421`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:29.165945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f5af0a10-958f-4848-8a07-d2b311fb4042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:29.183039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:29.272033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1277/2567] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:29.277083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3742ab0-82c3-4d5a-b76a-52f047470e4b admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:29.366901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3742ab0-82c3-4d5a-b76a-52f047470e4b admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:29.373631 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af3f21-0cdb-49a3-8138-76a2590917b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:29.374326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af3f21-0cdb-49a3-8138-76a2590917b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.375115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af3f21-0cdb-49a3-8138-76a2590917b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.375694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af3f21-0cdb-49a3-8138-76a2590917b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.376545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af3f21-0cdb-49a3-8138-76a2590917b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:29.378892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af3f21-0cdb-49a3-8138-76a2590917b7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:29.410047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1291/2568] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:29 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:29.415235 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af0a10-958f-4848-8a07-d2b311fb4042 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:29.429564 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af0a10-958f-4848-8a07-d2b311fb4042 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:29.492228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876d1a-397b-48f8-88ca-a7b043c343f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:29.492992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876d1a-397b-48f8-88ca-a7b043c343f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.493341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876d1a-397b-48f8-88ca-a7b043c343f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.493708 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876d1a-397b-48f8-88ca-a7b043c343f5 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.495675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7876d1a-397b-48f8-88ca-a7b043c343f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:29.497173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7876d1a-397b-48f8-88ca-a7b043c343f5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:29.525637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1278/2569] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:29 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:29.528883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af3f21-0cdb-49a3-8138-76a2590917b7 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:29.540231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af3f21-0cdb-49a3-8138-76a2590917b7 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:29.546531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc2746-e1c9-4549-9b34-9fc38c03030a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:29.547907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc2746-e1c9-4549-9b34-9fc38c03030a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.548313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc2746-e1c9-4549-9b34-9fc38c03030a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.549238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc2746-e1c9-4549-9b34-9fc38c03030a admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.550303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc2746-e1c9-4549-9b34-9fc38c03030a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:29.555270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc2746-e1c9-4549-9b34-9fc38c03030a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:29.568566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1292/2570] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:59:29.572614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876d1a-397b-48f8-88ca-a7b043c343f5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:29.583421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876d1a-397b-48f8-88ca-a7b043c343f5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:29.588023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa859a87-e4d4-40dd-b0b8-c8fcd07c140c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:29.588893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859a87-e4d4-40dd-b0b8-c8fcd07c140c admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.589169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859a87-e4d4-40dd-b0b8-c8fcd07c140c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.589464 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa859a87-e4d4-40dd-b0b8-c8fcd07c140c admin admin] PATH_INFO: `/v3/projects/182226af82884fcc9f705c2dafe4f9dd/users/12ef7c56e2c945808bca52c356e96560/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.607011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa859a87-e4d4-40dd-b0b8-c8fcd07c140c admin admin] RBAC: Authorizing `identity:create_grant(project_id=182226af82884fcc9f705c2dafe4f9dd, user_id=12ef7c56e2c945808bca52c356e96560, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:29.609818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa859a87-e4d4-40dd-b0b8-c8fcd07c140c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:29.633697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc2746-e1c9-4549-9b34-9fc38c03030a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:29.633924 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1279/2571] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:29 2022] PUT /identity/v3/projects/182226af82884fcc9f705c2dafe4f9dd/users/12ef7c56e2c945808bca52c356e96560/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:29.644599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc2746-e1c9-4549-9b34-9fc38c03030a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:29.712531 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5acb6-8bfe-47b9-a68e-a597fec9b965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:29.714350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5acb6-8bfe-47b9-a68e-a597fec9b965 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.714619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5acb6-8bfe-47b9-a68e-a597fec9b965 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.717268 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5acb6-8bfe-47b9-a68e-a597fec9b965 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.722071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5acb6-8bfe-47b9-a68e-a597fec9b965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:29.724257 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5acb6-8bfe-47b9-a68e-a597fec9b965 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:29.754192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1293/2572] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:59:29.758925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa859a87-e4d4-40dd-b0b8-c8fcd07c140c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:29.770971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa859a87-e4d4-40dd-b0b8-c8fcd07c140c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:29.775779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fabdb4-7fb1-424b-941f-80a898616eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:29.776791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fabdb4-7fb1-424b-941f-80a898616eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.777142 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fabdb4-7fb1-424b-941f-80a898616eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.777488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fabdb4-7fb1-424b-941f-80a898616eaf admin admin] PATH_INFO: `/v3/projects/182226af82884fcc9f705c2dafe4f9dd/users/12ef7c56e2c945808bca52c356e96560/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.779746 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fabdb4-7fb1-424b-941f-80a898616eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=182226af82884fcc9f705c2dafe4f9dd, user_id=12ef7c56e2c945808bca52c356e96560, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:29.782681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fabdb4-7fb1-424b-941f-80a898616eaf admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:29.806891 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466e30f-025b-47e0-bff5-d3ded962a821 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:29.807273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466e30f-025b-47e0-bff5-d3ded962a821 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:29.807627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466e30f-025b-47e0-bff5-d3ded962a821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:29.809504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1280/2573] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:29 2022] PUT /identity/v3/projects/182226af82884fcc9f705c2dafe4f9dd/users/12ef7c56e2c945808bca52c356e96560/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:59:29.854836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-d466e30f-025b-47e0-bff5-d3ded962a821 None None] MFA Rules not processed for user `12ef7c56e2c945808bca52c356e96560`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:29.856243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d466e30f-025b-47e0-bff5-d3ded962a821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:29.870347 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:29.953059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1294/2574] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:29.971427 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fabdb4-7fb1-424b-941f-80a898616eaf admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:29.974405 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f0fabdb4-7fb1-424b-941f-80a898616eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:30.079199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fabdb4-7fb1-424b-941f-80a898616eaf admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:30.083354 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e776e-a06c-4a8b-b3c9-bbeb8b80a601 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:30.084059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e776e-a06c-4a8b-b3c9-bbeb8b80a601 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:30.095247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e776e-a06c-4a8b-b3c9-bbeb8b80a601 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:30.095703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e776e-a06c-4a8b-b3c9-bbeb8b80a601 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:30.118469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e776e-a06c-4a8b-b3c9-bbeb8b80a601 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:30.120160 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e776e-a06c-4a8b-b3c9-bbeb8b80a601 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:30.134795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1281/2575] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:30.264014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466e30f-025b-47e0-bff5-d3ded962a821 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:30.310898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466e30f-025b-47e0-bff5-d3ded962a821 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:30.329442 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e972d2-63c7-49e9-b80c-d06359e4b3b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:30.330338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e972d2-63c7-49e9-b80c-d06359e4b3b2 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:30.330594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e972d2-63c7-49e9-b80c-d06359e4b3b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:30.330868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e972d2-63c7-49e9-b80c-d06359e4b3b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:30.488015 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e972d2-63c7-49e9-b80c-d06359e4b3b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:30.488686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e972d2-63c7-49e9-b80c-d06359e4b3b2 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:30.508971 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1295/2576] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:31.749390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e776e-a06c-4a8b-b3c9-bbeb8b80a601 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:31.752849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ff2e776e-a06c-4a8b-b3c9-bbeb8b80a601 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:31.777254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:31.840795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e776e-a06c-4a8b-b3c9-bbeb8b80a601 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:31.845363 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:31.846304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:31.846584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:31.846855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] PATH_INFO: `/v3/users/c815ea64d44a48d89a2f5ff10f3f9ee8` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:31.848938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c815ea64d44a48d89a2f5ff10f3f9ee8)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:31.850974 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:31.913176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c815ea64d44a48d89a2f5ff10f3f9ee8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:31.924448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c815ea64d44a48d89a2f5ff10f3f9ee8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:31.932908 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c815ea64d44a48d89a2f5ff10f3f9ee8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:31.940535 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c815ea64d44a48d89a2f5ff10f3f9ee8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:31.941565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c815ea64d44a48d89a2f5ff10f3f9ee8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:31.947928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1282/2577] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:31 2022] DELETE /identity/v3/users/c815ea64d44a48d89a2f5ff10f3f9ee8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:32.155068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e972d2-63c7-49e9-b80c-d06359e4b3b2 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:32.181898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e972d2-63c7-49e9-b80c-d06359e4b3b2 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:32.257205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:32.258121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:32.258398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:32.258689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] PATH_INFO: `/v3/projects/669587fe364c4741b489449451f2294c` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:32.259960 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=669587fe364c4741b489449451f2294c)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:32.261913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:32.320139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:32.323437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '669587fe364c4741b489449451f2294c', 'request_id': 'req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.333256 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '669587fe364c4741b489449451f2294c', 'request_id': 'req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.340470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '669587fe364c4741b489449451f2294c', 'request_id': 'req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.352088 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] The token cache is being invalidate because project 669587fe364c4741b489449451f2294c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:32.352383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.353125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.354712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1296/2578] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:32 2022] DELETE /identity/v3/projects/669587fe364c4741b489449451f2294c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:32.358660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:32.360027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:32.374503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd39b91-2ea4-4194-ae3f-bb76cb7d0539 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:32.434002 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:32.434791 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:32.435066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:32.435337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] PATH_INFO: `/v3/users/eef39c3815b84babb474d2e841f08831` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:32.436552 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef39c3815b84babb474d2e841f08831)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:32.438527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:32.476823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:32.493342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef39c3815b84babb474d2e841f08831'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.506111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef39c3815b84babb474d2e841f08831'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.513381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef39c3815b84babb474d2e841f08831'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.521153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef39c3815b84babb474d2e841f08831'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.521823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef39c3815b84babb474d2e841f08831'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.527679 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1283/2579] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:32 2022] DELETE /identity/v3/users/eef39c3815b84babb474d2e841f08831 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:32.699232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:32.700665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:32.715396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39cfe2-02f3-487f-9fe8-446f6d96a8c8 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:32.772567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:32.773423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:32.773739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:32.774011 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] PATH_INFO: `/v3/projects/43c3f9cd6a0b43eaad0a105608a22c3f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:32.775178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c3f9cd6a0b43eaad0a105608a22c3f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:32.777071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:32.822226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c3f9cd6a0b43eaad0a105608a22c3f', 'request_id': 'req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.829326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c3f9cd6a0b43eaad0a105608a22c3f', 'request_id': 'req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.835433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c3f9cd6a0b43eaad0a105608a22c3f', 'request_id': 'req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.845756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] The token cache is being invalidate because project 43c3f9cd6a0b43eaad0a105608a22c3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:32.846043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.846672 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b30ce883-6c7d-4bfa-b16d-dfc886f3c327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:32.848334 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1297/2580] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:32 2022] DELETE /identity/v3/projects/43c3f9cd6a0b43eaad0a105608a22c3f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:35.310596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:35.312077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:35.325621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5aa332-c143-488a-8da8-b99cad6b3616 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:35.388524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:35.389466 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:35.389782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:35.390058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] PATH_INFO: `/v3/users/7c31befc1f9f46cfa4f77a08db763421` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:35.391253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c31befc1f9f46cfa4f77a08db763421)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:35.393177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:35.454664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c31befc1f9f46cfa4f77a08db763421'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:35.466560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c31befc1f9f46cfa4f77a08db763421'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:35.474069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c31befc1f9f46cfa4f77a08db763421'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:35.480935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c31befc1f9f46cfa4f77a08db763421'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:35.481562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c31befc1f9f46cfa4f77a08db763421'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:35.487619 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1284/2581] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:35 2022] DELETE /identity/v3/users/7c31befc1f9f46cfa4f77a08db763421 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:35.492734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0738b-74ef-41bf-a6de-405ef2209d05 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:35.493224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0738b-74ef-41bf-a6de-405ef2209d05 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:35.493515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0738b-74ef-41bf-a6de-405ef2209d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:35.511640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-28d0738b-74ef-41bf-a6de-405ef2209d05 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:35.512847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-28d0738b-74ef-41bf-a6de-405ef2209d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:35.571100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1298/2582] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:35.580653 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:35.582199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:35.614530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:35.643911 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9a000-4c66-4b2a-b1ba-4897dccfeb77 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:35.647973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248c738-d778-4c88-8a5e-e89077e6cf1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:35.648614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c738-d778-4c88-8a5e-e89077e6cf1c service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:35.648897 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c738-d778-4c88-8a5e-e89077e6cf1c service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:35.649174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c738-d778-4c88-8a5e-e89077e6cf1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:35.658243 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248c738-d778-4c88-8a5e-e89077e6cf1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:35.660352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248c738-d778-4c88-8a5e-e89077e6cf1c service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:35.670067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1285/2583] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:35.946599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0738b-74ef-41bf-a6de-405ef2209d05 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:35.948044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-28d0738b-74ef-41bf-a6de-405ef2209d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:35.956443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:35.961725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0738b-74ef-41bf-a6de-405ef2209d05 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:35.968365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:35.969478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:35.969789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:35.970067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] PATH_INFO: `/v3/projects/a43d5410b59d4b17ac00aaefb4a2f1ff` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:35.971262 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a43d5410b59d4b17ac00aaefb4a2f1ff)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:35.973798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:36.020459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a43d5410b59d4b17ac00aaefb4a2f1ff', 'request_id': 'req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.027558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a43d5410b59d4b17ac00aaefb4a2f1ff', 'request_id': 'req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.033746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a43d5410b59d4b17ac00aaefb4a2f1ff', 'request_id': 'req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.044010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] The token cache is being invalidate because project a43d5410b59d4b17ac00aaefb4a2f1ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:36.044296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.044923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.046491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1299/2584] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:35 2022] DELETE /identity/v3/projects/a43d5410b59d4b17ac00aaefb4a2f1ff => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:36.051016 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248c738-d778-4c88-8a5e-e89077e6cf1c service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:36.060565 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248c738-d778-4c88-8a5e-e89077e6cf1c service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:36.119560 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:36.120450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:36.120737 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:36.121014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] PATH_INFO: `/v3/users/12ef7c56e2c945808bca52c356e96560` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:36.122411 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] RBAC: Authorizing `identity:delete_user(user_id=12ef7c56e2c945808bca52c356e96560)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:36.124460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:36.173106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12ef7c56e2c945808bca52c356e96560'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.185845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12ef7c56e2c945808bca52c356e96560'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.193498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12ef7c56e2c945808bca52c356e96560'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.200486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12ef7c56e2c945808bca52c356e96560'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.201115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-e026bf49-c47b-443a-8e96-cee43984a40c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12ef7c56e2c945808bca52c356e96560'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.207302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1286/2585] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:36 2022] DELETE /identity/v3/users/12ef7c56e2c945808bca52c356e96560 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:36.490294 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:36.492000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:36.507372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca8ff8-dc24-4585-a4a1-99b703cd3a2c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:36.576686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:36.577554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:36.577859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:36.578130 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] PATH_INFO: `/v3/projects/182226af82884fcc9f705c2dafe4f9dd` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:36.579301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] RBAC: Authorizing `identity:delete_project(project_id=182226af82884fcc9f705c2dafe4f9dd)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:36.581192 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:36.630007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182226af82884fcc9f705c2dafe4f9dd', 'request_id': 'req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.637392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182226af82884fcc9f705c2dafe4f9dd', 'request_id': 'req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.643832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182226af82884fcc9f705c2dafe4f9dd', 'request_id': 'req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.654501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] The token cache is being invalidate because project 182226af82884fcc9f705c2dafe4f9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:36.654782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.655403 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2d11f83d-bde8-4626-8fb1-0d26ba60e5be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:36.656921 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1300/2586] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:36 2022] DELETE /identity/v3/projects/182226af82884fcc9f705c2dafe4f9dd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:36.661943 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723be19-9df2-4560-9789-719277114b02 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:36.662458 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723be19-9df2-4560-9789-719277114b02 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:36.662563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723be19-9df2-4560-9789-719277114b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:36.680762 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-5723be19-9df2-4560-9789-719277114b02 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:36.682006 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5723be19-9df2-4560-9789-719277114b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:36.729754 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:36.746856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1287/2587] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:36.755689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92aad6-fb8f-4925-bfdd-c3e686cca714 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:36.755942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92aad6-fb8f-4925-bfdd-c3e686cca714 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:36.756328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92aad6-fb8f-4925-bfdd-c3e686cca714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:36.775870 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-0a92aad6-fb8f-4925-bfdd-c3e686cca714 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:36.777081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0a92aad6-fb8f-4925-bfdd-c3e686cca714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:36.785963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1301/2588] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:36.790561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723be19-9df2-4560-9789-719277114b02 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:36.802461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723be19-9df2-4560-9789-719277114b02 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:36.807083 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65bedd-3f09-49cd-b3f0-f156aa999f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:36.807747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65bedd-3f09-49cd-b3f0-f156aa999f1b admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:36.808030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65bedd-3f09-49cd-b3f0-f156aa999f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:36.808309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65bedd-3f09-49cd-b3f0-f156aa999f1b admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:36.809042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65bedd-3f09-49cd-b3f0-f156aa999f1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:59:36.809349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65bedd-3f09-49cd-b3f0-f156aa999f1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:36.811035 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65bedd-3f09-49cd-b3f0-f156aa999f1b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:36.824095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1288/2589] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 19:59:36.828410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92aad6-fb8f-4925-bfdd-c3e686cca714 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:36.837841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92aad6-fb8f-4925-bfdd-c3e686cca714 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:36.842596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b271e1f-7933-435b-bd8f-eebeae877936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:36.843319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b271e1f-7933-435b-bd8f-eebeae877936 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:36.843667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b271e1f-7933-435b-bd8f-eebeae877936 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:36.843947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b271e1f-7933-435b-bd8f-eebeae877936 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:36.844634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b271e1f-7933-435b-bd8f-eebeae877936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:36.846567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b271e1f-7933-435b-bd8f-eebeae877936 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:36.854915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:36.881064 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1302/2590] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:36 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:36.886524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65bedd-3f09-49cd-b3f0-f156aa999f1b admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:36.899577 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65bedd-3f09-49cd-b3f0-f156aa999f1b admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:36.961642 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ad83e-57f2-444c-893c-670383514e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:36.962235 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ad83e-57f2-444c-893c-670383514e64 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:36.962515 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ad83e-57f2-444c-893c-670383514e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:36.962814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ad83e-57f2-444c-893c-670383514e64 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:36.963524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ad83e-57f2-444c-893c-670383514e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:36.965475 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ad83e-57f2-444c-893c-670383514e64 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:36.991382 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1289/2591] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:36 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:59:36.992586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b271e1f-7933-435b-bd8f-eebeae877936 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:37.004086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b271e1f-7933-435b-bd8f-eebeae877936 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:37.009746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699816fd-b04c-4707-bfe1-5c74cba16876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:37.010590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699816fd-b04c-4707-bfe1-5c74cba16876 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:37.011140 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699816fd-b04c-4707-bfe1-5c74cba16876 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:37.011490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699816fd-b04c-4707-bfe1-5c74cba16876 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:37.012273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699816fd-b04c-4707-bfe1-5c74cba16876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:37.013913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699816fd-b04c-4707-bfe1-5c74cba16876 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:37.024482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1303/2592] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:37.028787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ad83e-57f2-444c-893c-670383514e64 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:37.039845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ad83e-57f2-444c-893c-670383514e64 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:37.044479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf3d46-fa50-4aca-9047-cdaeaa2746c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:37.045139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf3d46-fa50-4aca-9047-cdaeaa2746c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:37.045425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf3d46-fa50-4aca-9047-cdaeaa2746c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:37.045742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf3d46-fa50-4aca-9047-cdaeaa2746c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:37.046468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf3d46-fa50-4aca-9047-cdaeaa2746c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:37.048080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf3d46-fa50-4aca-9047-cdaeaa2746c9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:37.058882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1290/2593] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:59:37.063060 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699816fd-b04c-4707-bfe1-5c74cba16876 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:37.073091 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699816fd-b04c-4707-bfe1-5c74cba16876 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:37.077754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af736f-ca45-4f6f-a396-1144e6b54182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:37.078643 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af736f-ca45-4f6f-a396-1144e6b54182 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:37.078926 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af736f-ca45-4f6f-a396-1144e6b54182 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:37.079202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af736f-ca45-4f6f-a396-1144e6b54182 admin admin] PATH_INFO: `/v3/projects/bd147e07f5194a8da5b4b2bb27ce162e/users/afd73e45b9e140b8b7dd21552404dbbf/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:37.094922 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af736f-ca45-4f6f-a396-1144e6b54182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd147e07f5194a8da5b4b2bb27ce162e, user_id=afd73e45b9e140b8b7dd21552404dbbf, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:37.098169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af736f-ca45-4f6f-a396-1144e6b54182 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:37.112202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1304/2594] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:37 2022] PUT /identity/v3/projects/bd147e07f5194a8da5b4b2bb27ce162e/users/afd73e45b9e140b8b7dd21552404dbbf/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:37.118082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ace434-7c86-4d8f-a377-babda0879631 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:37.118367 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ace434-7c86-4d8f-a377-babda0879631 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:37.118638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ace434-7c86-4d8f-a377-babda0879631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:37.137023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-b6ace434-7c86-4d8f-a377-babda0879631 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:37.138140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b6ace434-7c86-4d8f-a377-babda0879631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:37.199296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1291/2595] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:37.209058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af736f-ca45-4f6f-a396-1144e6b54182 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:37.210778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d7af736f-ca45-4f6f-a396-1144e6b54182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:37.267657 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af736f-ca45-4f6f-a396-1144e6b54182 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:37.272302 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01f4e3-ee33-42cf-ad79-2e3caf3604dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:37.272943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01f4e3-ee33-42cf-ad79-2e3caf3604dc service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:37.273222 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01f4e3-ee33-42cf-ad79-2e3caf3604dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:37.273495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01f4e3-ee33-42cf-ad79-2e3caf3604dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:37.282573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01f4e3-ee33-42cf-ad79-2e3caf3604dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:37.284023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01f4e3-ee33-42cf-ad79-2e3caf3604dc service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:37.296085 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1305/2596] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:37.973933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2b9f3-4f09-46f3-a0e4-b27627ed07bb None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:37.974480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2b9f3-4f09-46f3-a0e4-b27627ed07bb None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:37.974480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2b9f3-4f09-46f3-a0e4-b27627ed07bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:38.001555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-57b2b9f3-4f09-46f3-a0e4-b27627ed07bb None None] MFA Rules not processed for user `afd73e45b9e140b8b7dd21552404dbbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:38.002802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-57b2b9f3-4f09-46f3-a0e4-b27627ed07bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:38.048505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:38.073062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1292/2597] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:38.077767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01f4e3-ee33-42cf-ad79-2e3caf3604dc service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:38.087945 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01f4e3-ee33-42cf-ad79-2e3caf3604dc service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:38.092470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9592c-5e91-4007-a3fb-0fcc116869f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:38.093114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9592c-5e91-4007-a3fb-0fcc116869f6 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:38.093392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9592c-5e91-4007-a3fb-0fcc116869f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:38.093689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9592c-5e91-4007-a3fb-0fcc116869f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:38.094380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9592c-5e91-4007-a3fb-0fcc116869f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:38.096269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9592c-5e91-4007-a3fb-0fcc116869f6 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:38.123111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1306/2598] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:38 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:38.127000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2b9f3-4f09-46f3-a0e4-b27627ed07bb None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:38.138746 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2b9f3-4f09-46f3-a0e4-b27627ed07bb None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:38.196607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035176d4-6729-4702-84d0-eb35f3f60655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:38.197274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035176d4-6729-4702-84d0-eb35f3f60655 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:38.197557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035176d4-6729-4702-84d0-eb35f3f60655 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:38.197862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035176d4-6729-4702-84d0-eb35f3f60655 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:38.198561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035176d4-6729-4702-84d0-eb35f3f60655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:38.200480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035176d4-6729-4702-84d0-eb35f3f60655 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:38.220702 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1293/2599] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:38 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:38.224555 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9592c-5e91-4007-a3fb-0fcc116869f6 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:38.235312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9592c-5e91-4007-a3fb-0fcc116869f6 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:38.241066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae524-f54a-4648-ac53-8e228ad61e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:38.241750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae524-f54a-4648-ac53-8e228ad61e70 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:38.242028 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae524-f54a-4648-ac53-8e228ad61e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:38.242296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae524-f54a-4648-ac53-8e228ad61e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:38.243010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecae524-f54a-4648-ac53-8e228ad61e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:38.244595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecae524-f54a-4648-ac53-8e228ad61e70 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:38.255929 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1307/2600] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:59:38.259980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035176d4-6729-4702-84d0-eb35f3f60655 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:38.270655 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035176d4-6729-4702-84d0-eb35f3f60655 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:38.275267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dc103-d7d2-4506-9098-769e3fa80d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:38.276132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dc103-d7d2-4506-9098-769e3fa80d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:38.276408 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dc103-d7d2-4506-9098-769e3fa80d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:38.276677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dc103-d7d2-4506-9098-769e3fa80d6b admin admin] PATH_INFO: `/v3/projects/a3b71f1fcadf46d48ba2c429064a5a40/users/ae018b31072442b9b937e1e76d6d58b3/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:38.292585 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083dc103-d7d2-4506-9098-769e3fa80d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b71f1fcadf46d48ba2c429064a5a40, user_id=ae018b31072442b9b937e1e76d6d58b3, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:38.295729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083dc103-d7d2-4506-9098-769e3fa80d6b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:38.308870 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1294/2601] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:38 2022] PUT /identity/v3/projects/a3b71f1fcadf46d48ba2c429064a5a40/users/ae018b31072442b9b937e1e76d6d58b3/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 19:59:38.312744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae524-f54a-4648-ac53-8e228ad61e70 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:38.323369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae524-f54a-4648-ac53-8e228ad61e70 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:38.378235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38508871-2428-4e60-9a27-9d92c96a788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:38.378925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38508871-2428-4e60-9a27-9d92c96a788e admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:38.379695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38508871-2428-4e60-9a27-9d92c96a788e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:38.379968 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38508871-2428-4e60-9a27-9d92c96a788e admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:38.380684 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38508871-2428-4e60-9a27-9d92c96a788e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:38.382311 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38508871-2428-4e60-9a27-9d92c96a788e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:38.391887 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1308/2602] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:38.396260 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dc103-d7d2-4506-9098-769e3fa80d6b admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:38.406519 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dc103-d7d2-4506-9098-769e3fa80d6b admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:38.411638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ac2fa-8f0a-4ff8-94ab-561ffbd3361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:38.412511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ac2fa-8f0a-4ff8-94ab-561ffbd3361a admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:38.412799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ac2fa-8f0a-4ff8-94ab-561ffbd3361a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:38.413086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ac2fa-8f0a-4ff8-94ab-561ffbd3361a admin admin] PATH_INFO: `/v3/projects/a3b71f1fcadf46d48ba2c429064a5a40/users/ae018b31072442b9b937e1e76d6d58b3/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:38.415074 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ac2fa-8f0a-4ff8-94ab-561ffbd3361a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b71f1fcadf46d48ba2c429064a5a40, user_id=ae018b31072442b9b937e1e76d6d58b3, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:38.417719 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ac2fa-8f0a-4ff8-94ab-561ffbd3361a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:38.428449 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1295/2603] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:38 2022] PUT /identity/v3/projects/a3b71f1fcadf46d48ba2c429064a5a40/users/ae018b31072442b9b937e1e76d6d58b3/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:39.171514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172505-31bf-435d-9446-ded235dad74d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:39.172056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172505-31bf-435d-9446-ded235dad74d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:39.172056 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172505-31bf-435d-9446-ded235dad74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:39.205742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-ed172505-31bf-435d-9446-ded235dad74d None None] MFA Rules not processed for user `ae018b31072442b9b937e1e76d6d58b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:39.207733 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ed172505-31bf-435d-9446-ded235dad74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:39.216577 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:39.321451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1309/2604] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:39.334142 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ac2fa-8f0a-4ff8-94ab-561ffbd3361a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:39.337208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e35ac2fa-8f0a-4ff8-94ab-561ffbd3361a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:39.419826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ac2fa-8f0a-4ff8-94ab-561ffbd3361a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:39.430306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee7fd8-f300-46ea-b762-061c877a5694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:39.431940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee7fd8-f300-46ea-b762-061c877a5694 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:39.433758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee7fd8-f300-46ea-b762-061c877a5694 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:39.434960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee7fd8-f300-46ea-b762-061c877a5694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:39.502066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:39.509550 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee7fd8-f300-46ea-b762-061c877a5694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:39.511147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee7fd8-f300-46ea-b762-061c877a5694 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:39.524975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1296/2605] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:40.888027 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed172505-31bf-435d-9446-ded235dad74d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:40.889480 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ed172505-31bf-435d-9446-ded235dad74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:40.901871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed172505-31bf-435d-9446-ded235dad74d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:40.905706 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf9be-c915-463d-8ffd-2ab3809161f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:40.906732 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf9be-c915-463d-8ffd-2ab3809161f0 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:40.906732 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf9be-c915-463d-8ffd-2ab3809161f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:40.907111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbf9be-c915-463d-8ffd-2ab3809161f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:40.919230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbf9be-c915-463d-8ffd-2ab3809161f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:40.920746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbf9be-c915-463d-8ffd-2ab3809161f0 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:40.931710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1310/2606] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:48.273286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee7fd8-f300-46ea-b762-061c877a5694 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:48.328410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee7fd8-f300-46ea-b762-061c877a5694 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:48.332026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a437f-8b2f-4be6-84c2-cf3e56d77992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:48.332726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a437f-8b2f-4be6-84c2-cf3e56d77992 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:48.333136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a437f-8b2f-4be6-84c2-cf3e56d77992 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:48.333397 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a437f-8b2f-4be6-84c2-cf3e56d77992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:48.335246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4a5a437f-8b2f-4be6-84c2-cf3e56d77992 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:48.398705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a437f-8b2f-4be6-84c2-cf3e56d77992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:48.400140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a437f-8b2f-4be6-84c2-cf3e56d77992 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:48.409837 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1297/2607] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:50.927247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=31 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:51.662330 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf9be-c915-463d-8ffd-2ab3809161f0 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:51.663799 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-35cbf9be-c915-463d-8ffd-2ab3809161f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:51.678774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbf9be-c915-463d-8ffd-2ab3809161f0 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:51.683402 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:51.684348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:51.684768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:51.685112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] PATH_INFO: `/v3/users/3307be3159654f55be46e75c82584bf0` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:51.686499 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3307be3159654f55be46e75c82584bf0)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:51.688476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:51.738463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3307be3159654f55be46e75c82584bf0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:51.748438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3307be3159654f55be46e75c82584bf0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:51.749241 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3307be3159654f55be46e75c82584bf0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:51.762014 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3307be3159654f55be46e75c82584bf0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:51.762665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-3f3a18c8-99a4-494d-b911-3f05e2acd581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3307be3159654f55be46e75c82584bf0'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:51.778534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1311/2608] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:51 2022] DELETE /identity/v3/users/3307be3159654f55be46e75c82584bf0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:51.962338 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a437f-8b2f-4be6-84c2-cf3e56d77992 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:51.972861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a437f-8b2f-4be6-84c2-cf3e56d77992 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.036214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.037154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.037435 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.037748 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] PATH_INFO: `/v3/projects/b8e38ec3f92a47fe98829eb3d289cdb4` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.038948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8e38ec3f92a47fe98829eb3d289cdb4)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.040871 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.087247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:52.090698 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8e38ec3f92a47fe98829eb3d289cdb4', 'request_id': 'req-07c3d384-5730-4eff-be39-1eb07eea727e'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:52.099122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8e38ec3f92a47fe98829eb3d289cdb4', 'request_id': 'req-07c3d384-5730-4eff-be39-1eb07eea727e'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:52.104785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8e38ec3f92a47fe98829eb3d289cdb4', 'request_id': 'req-07c3d384-5730-4eff-be39-1eb07eea727e'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:52.116031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] The token cache is being invalidate because project b8e38ec3f92a47fe98829eb3d289cdb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:52.116318 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:52.116959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-07c3d384-5730-4eff-be39-1eb07eea727e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:52.118540 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1298/2609] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:51 2022] DELETE /identity/v3/projects/b8e38ec3f92a47fe98829eb3d289cdb4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:52.123569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f30cc6-be29-4299-bbac-bb8ae7fc326b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.124029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f30cc6-be29-4299-bbac-bb8ae7fc326b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.124139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f30cc6-be29-4299-bbac-bb8ae7fc326b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.142947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a2f30cc6-be29-4299-bbac-bb8ae7fc326b None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:52.144148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a2f30cc6-be29-4299-bbac-bb8ae7fc326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:52.202508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1312/2610] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:52.211182 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc61cd5-b0ac-4243-ab26-3a8eed740fa3 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.211468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc61cd5-b0ac-4243-ab26-3a8eed740fa3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.211747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc61cd5-b0ac-4243-ab26-3a8eed740fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.230676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-3fc61cd5-b0ac-4243-ab26-3a8eed740fa3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:52.231865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3fc61cd5-b0ac-4243-ab26-3a8eed740fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:52.240137 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1299/2611] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:52.244143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f30cc6-be29-4299-bbac-bb8ae7fc326b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.253640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f30cc6-be29-4299-bbac-bb8ae7fc326b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.259538 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0275c6-cbd8-440f-931a-c9fb2e6e3ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.260189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0275c6-cbd8-440f-931a-c9fb2e6e3ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.260466 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0275c6-cbd8-440f-931a-c9fb2e6e3ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.260736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0275c6-cbd8-440f-931a-c9fb2e6e3ba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.261450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0275c6-cbd8-440f-931a-c9fb2e6e3ba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:59:52.261783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0275c6-cbd8-440f-931a-c9fb2e6e3ba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.263365 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0275c6-cbd8-440f-931a-c9fb2e6e3ba2 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.275992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1313/2612] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:59:52.280034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc61cd5-b0ac-4243-ab26-3a8eed740fa3 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.290468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc61cd5-b0ac-4243-ab26-3a8eed740fa3 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.296050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab723f-40c0-40d5-820f-ba3541b65455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.296700 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab723f-40c0-40d5-820f-ba3541b65455 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.296983 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab723f-40c0-40d5-820f-ba3541b65455 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.297258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab723f-40c0-40d5-820f-ba3541b65455 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.297998 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab723f-40c0-40d5-820f-ba3541b65455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.299884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab723f-40c0-40d5-820f-ba3541b65455 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.324764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1300/2613] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:52 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:59:52.328700 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0275c6-cbd8-440f-931a-c9fb2e6e3ba2 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.339683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0275c6-cbd8-440f-931a-c9fb2e6e3ba2 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.395513 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be23ce-36f1-40f0-acb0-92965ed778f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.396188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be23ce-36f1-40f0-acb0-92965ed778f3 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.396467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be23ce-36f1-40f0-acb0-92965ed778f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.396737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be23ce-36f1-40f0-acb0-92965ed778f3 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.397424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be23ce-36f1-40f0-acb0-92965ed778f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.399356 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be23ce-36f1-40f0-acb0-92965ed778f3 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.405999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:52.421719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1314/2614] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:52 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:52.425839 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab723f-40c0-40d5-820f-ba3541b65455 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.436150 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab723f-40c0-40d5-820f-ba3541b65455 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.440775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1823bf-441e-41ae-b6a8-18b61e662023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.441428 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1823bf-441e-41ae-b6a8-18b61e662023 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.442246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1823bf-441e-41ae-b6a8-18b61e662023 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.442557 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1823bf-441e-41ae-b6a8-18b61e662023 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.443349 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1823bf-441e-41ae-b6a8-18b61e662023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.444945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1823bf-441e-41ae-b6a8-18b61e662023 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.455202 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1301/2615] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:52.459915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be23ce-36f1-40f0-acb0-92965ed778f3 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.469786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be23ce-36f1-40f0-acb0-92965ed778f3 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.474573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f92bef-e697-4125-9be4-7725931ca3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.475281 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f92bef-e697-4125-9be4-7725931ca3dd admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.475562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f92bef-e697-4125-9be4-7725931ca3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.475826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f92bef-e697-4125-9be4-7725931ca3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.476609 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f92bef-e697-4125-9be4-7725931ca3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.478223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f92bef-e697-4125-9be4-7725931ca3dd admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.488885 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1315/2616] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:59:52.493539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1823bf-441e-41ae-b6a8-18b61e662023 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.503480 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1823bf-441e-41ae-b6a8-18b61e662023 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.508819 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf49b43-575a-4e52-9620-b200b03df718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.509736 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf49b43-575a-4e52-9620-b200b03df718 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.510026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf49b43-575a-4e52-9620-b200b03df718 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.510303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf49b43-575a-4e52-9620-b200b03df718 admin admin] PATH_INFO: `/v3/projects/4b0163c13c794461b892e18258b5d08b/users/1255c3a12e0b44ff934d2dd19669100a/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.526386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf49b43-575a-4e52-9620-b200b03df718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0163c13c794461b892e18258b5d08b, user_id=1255c3a12e0b44ff934d2dd19669100a, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.529027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf49b43-575a-4e52-9620-b200b03df718 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.541662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1302/2617] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:52 2022] PUT /identity/v3/projects/4b0163c13c794461b892e18258b5d08b/users/1255c3a12e0b44ff934d2dd19669100a/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:52.552103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac73da-736c-4bc5-9b60-a924fd0f755b None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.552463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac73da-736c-4bc5-9b60-a924fd0f755b None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.552856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac73da-736c-4bc5-9b60-a924fd0f755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.579707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-e2ac73da-736c-4bc5-9b60-a924fd0f755b None None] MFA Rules not processed for user `1255c3a12e0b44ff934d2dd19669100a`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:52.580905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e2ac73da-736c-4bc5-9b60-a924fd0f755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:52.643407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1316/2618] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:52.648657 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf49b43-575a-4e52-9620-b200b03df718 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.709357 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf49b43-575a-4e52-9620-b200b03df718 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.714000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dfbd6-2523-4252-a3d6-0e1615c17390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.714650 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dfbd6-2523-4252-a3d6-0e1615c17390 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.714934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dfbd6-2523-4252-a3d6-0e1615c17390 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.715212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dfbd6-2523-4252-a3d6-0e1615c17390 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.715910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89dfbd6-2523-4252-a3d6-0e1615c17390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.717841 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89dfbd6-2523-4252-a3d6-0e1615c17390 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.742664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1303/2619] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:52 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:52.746754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac73da-736c-4bc5-9b60-a924fd0f755b None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.759213 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac73da-736c-4bc5-9b60-a924fd0f755b None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.815512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d16d4-c78c-42ad-9823-8938f907a02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.816191 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d16d4-c78c-42ad-9823-8938f907a02c admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.816487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d16d4-c78c-42ad-9823-8938f907a02c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.817358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d16d4-c78c-42ad-9823-8938f907a02c admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.818168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d16d4-c78c-42ad-9823-8938f907a02c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.820097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d16d4-c78c-42ad-9823-8938f907a02c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.839102 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1317/2620] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:52 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:52.843060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dfbd6-2523-4252-a3d6-0e1615c17390 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.853304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dfbd6-2523-4252-a3d6-0e1615c17390 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.857965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ba297-2b3c-4ff1-9383-e4aaa5f2cb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.858621 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba297-2b3c-4ff1-9383-e4aaa5f2cb7a admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.858903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba297-2b3c-4ff1-9383-e4aaa5f2cb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.859181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba297-2b3c-4ff1-9383-e4aaa5f2cb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.859904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ba297-2b3c-4ff1-9383-e4aaa5f2cb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.861503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ba297-2b3c-4ff1-9383-e4aaa5f2cb7a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.872465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1304/2621] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:52.876524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d16d4-c78c-42ad-9823-8938f907a02c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.887168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d16d4-c78c-42ad-9823-8938f907a02c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.891815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa874dee-b210-438f-9cf6-c15e3b64f310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:52.892694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa874dee-b210-438f-9cf6-c15e3b64f310 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:52.892973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa874dee-b210-438f-9cf6-c15e3b64f310 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:52.893243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa874dee-b210-438f-9cf6-c15e3b64f310 admin admin] PATH_INFO: `/v3/projects/dd5a72da634c45828b6983d093e23b4c/users/b71c9e74549e4745ace480ea0b5adf30/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:52.907996 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa874dee-b210-438f-9cf6-c15e3b64f310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd5a72da634c45828b6983d093e23b4c, user_id=b71c9e74549e4745ace480ea0b5adf30, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:52.910776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa874dee-b210-438f-9cf6-c15e3b64f310 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:52.923282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1318/2622] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:52 2022] PUT /identity/v3/projects/dd5a72da634c45828b6983d093e23b4c/users/b71c9e74549e4745ace480ea0b5adf30/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:52.927861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ba297-2b3c-4ff1-9383-e4aaa5f2cb7a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:52.938012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ba297-2b3c-4ff1-9383-e4aaa5f2cb7a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:52.982429 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:53.000391 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5d745-5b14-4fa8-8a73-21796c604e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:53.001059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5d745-5b14-4fa8-8a73-21796c604e47 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:53.001344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5d745-5b14-4fa8-8a73-21796c604e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:53.001648 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5d745-5b14-4fa8-8a73-21796c604e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:53.002375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5d745-5b14-4fa8-8a73-21796c604e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:53.004234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5d745-5b14-4fa8-8a73-21796c604e47 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:53.015803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1305/2623] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 19:59:53.020209 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa874dee-b210-438f-9cf6-c15e3b64f310 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:53.030612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa874dee-b210-438f-9cf6-c15e3b64f310 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:53.035279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768acaf1-80a7-4d39-a8b6-dfe744ba0d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:53.036297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768acaf1-80a7-4d39-a8b6-dfe744ba0d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:53.036509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768acaf1-80a7-4d39-a8b6-dfe744ba0d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:53.036773 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768acaf1-80a7-4d39-a8b6-dfe744ba0d9f admin admin] PATH_INFO: `/v3/projects/dd5a72da634c45828b6983d093e23b4c/users/b71c9e74549e4745ace480ea0b5adf30/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:53.039888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768acaf1-80a7-4d39-a8b6-dfe744ba0d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd5a72da634c45828b6983d093e23b4c, user_id=b71c9e74549e4745ace480ea0b5adf30, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:53.042557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768acaf1-80a7-4d39-a8b6-dfe744ba0d9f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:53.054384 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1319/2624] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:53 2022] PUT /identity/v3/projects/dd5a72da634c45828b6983d093e23b4c/users/b71c9e74549e4745ace480ea0b5adf30/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:53.065640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b34917-20ba-484d-880f-f411a218c608 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:53.065959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b34917-20ba-484d-880f-f411a218c608 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:53.066240 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b34917-20ba-484d-880f-f411a218c608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:53.093018 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-69b34917-20ba-484d-880f-f411a218c608 None None] MFA Rules not processed for user `b71c9e74549e4745ace480ea0b5adf30`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:53.094269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-69b34917-20ba-484d-880f-f411a218c608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:53.167996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1306/2625] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 19:59:53.181205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768acaf1-80a7-4d39-a8b6-dfe744ba0d9f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:53.183194 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-768acaf1-80a7-4d39-a8b6-dfe744ba0d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:53.206339 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:53.247202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768acaf1-80a7-4d39-a8b6-dfe744ba0d9f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:53.250854 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55140a-4f55-4124-abe3-0f571e7877f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:53.251562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55140a-4f55-4124-abe3-0f571e7877f5 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:53.251790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55140a-4f55-4124-abe3-0f571e7877f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:53.252063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55140a-4f55-4124-abe3-0f571e7877f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:53.262272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55140a-4f55-4124-abe3-0f571e7877f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:53.263726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55140a-4f55-4124-abe3-0f571e7877f5 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:53.275491 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1320/2626] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:53.351518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b34917-20ba-484d-880f-f411a218c608 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:53.360984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b34917-20ba-484d-880f-f411a218c608 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:53.365009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b824014-89ec-4dde-8cfa-d7df04b00d32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:53.365685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b824014-89ec-4dde-8cfa-d7df04b00d32 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:53.365972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b824014-89ec-4dde-8cfa-d7df04b00d32 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:53.366250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b824014-89ec-4dde-8cfa-d7df04b00d32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:53.431345 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b824014-89ec-4dde-8cfa-d7df04b00d32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:53.432860 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b824014-89ec-4dde-8cfa-d7df04b00d32 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:53.442677 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1307/2627] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 19:59:54.092506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55140a-4f55-4124-abe3-0f571e7877f5 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:54.094168 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ca55140a-4f55-4124-abe3-0f571e7877f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:54.159694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55140a-4f55-4124-abe3-0f571e7877f5 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:54.165152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:54.166062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:54.166332 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:54.166628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] PATH_INFO: `/v3/users/afd73e45b9e140b8b7dd21552404dbbf` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:54.167783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd73e45b9e140b8b7dd21552404dbbf)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:54.169714 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:54.219204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd73e45b9e140b8b7dd21552404dbbf'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.228498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd73e45b9e140b8b7dd21552404dbbf'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.229401 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd73e45b9e140b8b7dd21552404dbbf'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.242842 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd73e45b9e140b8b7dd21552404dbbf'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.243539 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd73e45b9e140b8b7dd21552404dbbf'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.256669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1321/2628] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:54 2022] DELETE /identity/v3/users/afd73e45b9e140b8b7dd21552404dbbf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:54.429122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b824014-89ec-4dde-8cfa-d7df04b00d32 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:54.441311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b824014-89ec-4dde-8cfa-d7df04b00d32 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:54.502818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:54.503679 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:54.503959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:54.504251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] PATH_INFO: `/v3/projects/bd147e07f5194a8da5b4b2bb27ce162e` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:54.505959 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd147e07f5194a8da5b4b2bb27ce162e)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:54.507956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:54.514710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:54.556893 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd147e07f5194a8da5b4b2bb27ce162e', 'request_id': 'req-ee560dae-da55-406d-aaf1-e18642d7c117'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.564138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd147e07f5194a8da5b4b2bb27ce162e', 'request_id': 'req-ee560dae-da55-406d-aaf1-e18642d7c117'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.570141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd147e07f5194a8da5b4b2bb27ce162e', 'request_id': 'req-ee560dae-da55-406d-aaf1-e18642d7c117'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.581005 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] The token cache is being invalidate because project bd147e07f5194a8da5b4b2bb27ce162e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:54.581291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.581952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.584147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1308/2629] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:54 2022] DELETE /identity/v3/projects/bd147e07f5194a8da5b4b2bb27ce162e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:54.588269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:54.589754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:54.602700 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98492e-843f-496e-bc81-8d79a5a9da25 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:54.646509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:54.664664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:54.665543 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:54.665853 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:54.666124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] PATH_INFO: `/v3/users/ae018b31072442b9b937e1e76d6d58b3` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:54.667301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae018b31072442b9b937e1e76d6d58b3)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:54.669198 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:54.719605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae018b31072442b9b937e1e76d6d58b3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.726739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae018b31072442b9b937e1e76d6d58b3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.727385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae018b31072442b9b937e1e76d6d58b3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.739645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae018b31072442b9b937e1e76d6d58b3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.740599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-49f7f8be-35fd-4b5c-9a2b-0ccdf313ca91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae018b31072442b9b937e1e76d6d58b3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:54.752370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1322/2630] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 19:59:54 2022] DELETE /identity/v3/users/ae018b31072442b9b937e1e76d6d58b3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:54.964001 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:54.966157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:54.984211 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee560dae-da55-406d-aaf1-e18642d7c117 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:55.049539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:55.050461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.050742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.051547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] PATH_INFO: `/v3/projects/a3b71f1fcadf46d48ba2c429064a5a40` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.052737 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3b71f1fcadf46d48ba2c429064a5a40)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:55.054674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:55.100090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3b71f1fcadf46d48ba2c429064a5a40', 'request_id': 'req-88145e45-14b4-4f2c-b46f-95452502aecc'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:55.106812 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3b71f1fcadf46d48ba2c429064a5a40', 'request_id': 'req-88145e45-14b4-4f2c-b46f-95452502aecc'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:55.112673 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3b71f1fcadf46d48ba2c429064a5a40', 'request_id': 'req-88145e45-14b4-4f2c-b46f-95452502aecc'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:55.124388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] The token cache is being invalidate because project a3b71f1fcadf46d48ba2c429064a5a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 19:59:55.124672 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:55.125304 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-88145e45-14b4-4f2c-b46f-95452502aecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 19:59:55.126932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1309/2631] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 19:59:54 2022] DELETE /identity/v3/projects/a3b71f1fcadf46d48ba2c429064a5a40 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:55.132495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab385d9-dae7-4772-9f0d-d21e15f5790d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.132816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab385d9-dae7-4772-9f0d-d21e15f5790d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.133101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab385d9-dae7-4772-9f0d-d21e15f5790d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.151476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-3ab385d9-dae7-4772-9f0d-d21e15f5790d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:55.152721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3ab385d9-dae7-4772-9f0d-d21e15f5790d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:55.215739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1323/2632] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:55.224380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364851fa-5caf-4e63-adf6-84ab0219a06f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.224794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364851fa-5caf-4e63-adf6-84ab0219a06f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.225077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364851fa-5caf-4e63-adf6-84ab0219a06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.244543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-364851fa-5caf-4e63-adf6-84ab0219a06f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:55.245813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-364851fa-5caf-4e63-adf6-84ab0219a06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:55.253923 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1310/2633] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:55.258743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab385d9-dae7-4772-9f0d-d21e15f5790d None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:55.268721 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab385d9-dae7-4772-9f0d-d21e15f5790d None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:55.273605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bc95a-3ccd-4376-a334-5f433e8bc239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:55.274265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bc95a-3ccd-4376-a334-5f433e8bc239 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.274547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bc95a-3ccd-4376-a334-5f433e8bc239 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.274822 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bc95a-3ccd-4376-a334-5f433e8bc239 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.275548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bc95a-3ccd-4376-a334-5f433e8bc239 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 19:59:55.275854 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bc95a-3ccd-4376-a334-5f433e8bc239 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:55.277457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bc95a-3ccd-4376-a334-5f433e8bc239 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:55.289018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1324/2634] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 19:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 19:59:55.294785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364851fa-5caf-4e63-adf6-84ab0219a06f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:55.304609 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364851fa-5caf-4e63-adf6-84ab0219a06f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:55.309275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd7128-ec2e-46e4-8c15-9b5a49073cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:55.309969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd7128-ec2e-46e4-8c15-9b5a49073cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.310250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd7128-ec2e-46e4-8c15-9b5a49073cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.310526 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd7128-ec2e-46e4-8c15-9b5a49073cd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.311208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd7128-ec2e-46e4-8c15-9b5a49073cd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:55.313293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd7128-ec2e-46e4-8c15-9b5a49073cd2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:55.328019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:55.341095 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1311/2635] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 19:59:55 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 19:59:55.345179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bc95a-3ccd-4376-a334-5f433e8bc239 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:55.355736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bc95a-3ccd-4376-a334-5f433e8bc239 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:55.400529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 19:59:55.414148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f9e3c-b59b-40a6-aa80-5aef7b148c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:55.414812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f9e3c-b59b-40a6-aa80-5aef7b148c21 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.415120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f9e3c-b59b-40a6-aa80-5aef7b148c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.415393 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f9e3c-b59b-40a6-aa80-5aef7b148c21 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.416113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f9e3c-b59b-40a6-aa80-5aef7b148c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:55.418070 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f9e3c-b59b-40a6-aa80-5aef7b148c21 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:55.437456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1325/2636] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 19:59:55 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 19:59:55.441115 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd7128-ec2e-46e4-8c15-9b5a49073cd2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:55.451421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd7128-ec2e-46e4-8c15-9b5a49073cd2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:55.455987 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef6226-f883-4653-a73b-8dee9a165697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:55.456634 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef6226-f883-4653-a73b-8dee9a165697 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.456922 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef6226-f883-4653-a73b-8dee9a165697 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.457296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef6226-f883-4653-a73b-8dee9a165697 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.458051 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef6226-f883-4653-a73b-8dee9a165697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:55.463184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef6226-f883-4653-a73b-8dee9a165697 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:55.475392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1312/2637] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:55.479542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f9e3c-b59b-40a6-aa80-5aef7b148c21 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:55.490983 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f9e3c-b59b-40a6-aa80-5aef7b148c21 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:55.495865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc28ac-d79e-4bc5-8d61-0a240fe66a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:55.496530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc28ac-d79e-4bc5-8d61-0a240fe66a93 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.496815 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc28ac-d79e-4bc5-8d61-0a240fe66a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.497090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc28ac-d79e-4bc5-8d61-0a240fe66a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.497851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc28ac-d79e-4bc5-8d61-0a240fe66a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:55.499518 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc28ac-d79e-4bc5-8d61-0a240fe66a93 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:55.510736 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1326/2638] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 19:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 19:59:55.515012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef6226-f883-4653-a73b-8dee9a165697 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:55.526062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef6226-f883-4653-a73b-8dee9a165697 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:55.530814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2694c3-6112-42b2-a467-f66b17b48873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:55.531694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2694c3-6112-42b2-a467-f66b17b48873 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.531978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2694c3-6112-42b2-a467-f66b17b48873 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.532261 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2694c3-6112-42b2-a467-f66b17b48873 admin admin] PATH_INFO: `/v3/projects/5b69790e7541443b9bf62c0df7128793/users/94a321be52654e819642f15706634f52/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.547545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2694c3-6112-42b2-a467-f66b17b48873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b69790e7541443b9bf62c0df7128793, user_id=94a321be52654e819642f15706634f52, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:55.550248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2694c3-6112-42b2-a467-f66b17b48873 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:55.564052 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1313/2639] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 19:59:55 2022] PUT /identity/v3/projects/5b69790e7541443b9bf62c0df7128793/users/94a321be52654e819642f15706634f52/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 19:59:55.575435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeea02c-7ed0-4a73-abc7-ee73087550ad None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.575725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeea02c-7ed0-4a73-abc7-ee73087550ad None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.576004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeea02c-7ed0-4a73-abc7-ee73087550ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.603569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-caeea02c-7ed0-4a73-abc7-ee73087550ad None None] MFA Rules not processed for user `94a321be52654e819642f15706634f52`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 19:59:55.604807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-caeea02c-7ed0-4a73-abc7-ee73087550ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:55.667896 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1327/2640] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 19:59:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 19:59:55.680030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2694c3-6112-42b2-a467-f66b17b48873 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 19:59:55.681822 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2b2694c3-6112-42b2-a467-f66b17b48873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 19:59:55.739939 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2694c3-6112-42b2-a467-f66b17b48873 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 19:59:55.743518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3728f-9122-4e8b-8bfc-0c43465fab3f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c022224b03a54ab489d95502c63a25dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 19:59:55.744154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3728f-9122-4e8b-8bfc-0c43465fab3f service glance] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 19:59:55.744452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3728f-9122-4e8b-8bfc-0c43465fab3f service glance] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 19:59:55.744709 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3728f-9122-4e8b-8bfc-0c43465fab3f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 19:59:55.752887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3728f-9122-4e8b-8bfc-0c43465fab3f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 19:59:55.754346 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3728f-9122-4e8b-8bfc-0c43465fab3f service glance] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 19:59:55.762972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1314/2641] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 19:59:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:00:05.665007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:05.759063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:13.393522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeea02c-7ed0-4a73-abc7-ee73087550ad None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:13.455360 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeea02c-7ed0-4a73-abc7-ee73087550ad None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:13.460074 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:13.461174 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:13.461458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:13.461766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] PATH_INFO: `/v3/users/94a321be52654e819642f15706634f52` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:13.462980 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a321be52654e819642f15706634f52)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:13.464882 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:13.515105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a321be52654e819642f15706634f52'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:13.523611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a321be52654e819642f15706634f52'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:13.524266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a321be52654e819642f15706634f52'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:13.538790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a321be52654e819642f15706634f52'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:13.539503 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a321be52654e819642f15706634f52'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:13.560321 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1328/2642] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:00:13 2022] DELETE /identity/v3/users/94a321be52654e819642f15706634f52 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:00:13.565185 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fab43f-a5b5-4a73-a9c6-c53e9b7bf4b5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:13.565697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fab43f-a5b5-4a73-a9c6-c53e9b7bf4b5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:13.566085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fab43f-a5b5-4a73-a9c6-c53e9b7bf4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:13.585174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-59fab43f-a5b5-4a73-a9c6-c53e9b7bf4b5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:13.586420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-59fab43f-a5b5-4a73-a9c6-c53e9b7bf4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:13.648586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1315/2643] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:13.658255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:13.660224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:13.716711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b49f4-0dc1-4ba1-87af-dd6820053918 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:13.720301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a77343-8755-43b3-99f4-915a329b8277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:13.720940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a77343-8755-43b3-99f4-915a329b8277 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:13.721223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a77343-8755-43b3-99f4-915a329b8277 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:13.721501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a77343-8755-43b3-99f4-915a329b8277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:13.731087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a77343-8755-43b3-99f4-915a329b8277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:13.732512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a77343-8755-43b3-99f4-915a329b8277 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:13.742037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1329/2644] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:00:14.013706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fab43f-a5b5-4a73-a9c6-c53e9b7bf4b5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:14.015128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-59fab43f-a5b5-4a73-a9c6-c53e9b7bf4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:14.027717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fab43f-a5b5-4a73-a9c6-c53e9b7bf4b5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:14.032329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:14.033311 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.033534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.033857 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] PATH_INFO: `/v3/projects/5b69790e7541443b9bf62c0df7128793` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.035824 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b69790e7541443b9bf62c0df7128793)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:14.037783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:14.085211 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b69790e7541443b9bf62c0df7128793', 'request_id': 'req-7736ca55-27aa-4fee-8ed9-d175004da1ad'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:14.092086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b69790e7541443b9bf62c0df7128793', 'request_id': 'req-7736ca55-27aa-4fee-8ed9-d175004da1ad'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:14.097746 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b69790e7541443b9bf62c0df7128793', 'request_id': 'req-7736ca55-27aa-4fee-8ed9-d175004da1ad'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:14.108422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] The token cache is being invalidate because project 5b69790e7541443b9bf62c0df7128793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:00:14.108706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:14.109322 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-7736ca55-27aa-4fee-8ed9-d175004da1ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:14.110898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1316/2645] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:00:14 2022] DELETE /identity/v3/projects/5b69790e7541443b9bf62c0df7128793 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:00:14.116419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d56bd9-406a-48e3-b926-5f0ac2d9f754 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.116709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d56bd9-406a-48e3-b926-5f0ac2d9f754 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.116990 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d56bd9-406a-48e3-b926-5f0ac2d9f754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.135664 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-94d56bd9-406a-48e3-b926-5f0ac2d9f754 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:14.137131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-94d56bd9-406a-48e3-b926-5f0ac2d9f754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:14.154418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:14.198723 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1330/2646] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:14.209644 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e2edb-f2a6-487d-9907-4f36349ee2f8 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.209934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e2edb-f2a6-487d-9907-4f36349ee2f8 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.210209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e2edb-f2a6-487d-9907-4f36349ee2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.229168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-ab3e2edb-f2a6-487d-9907-4f36349ee2f8 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:14.230412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ab3e2edb-f2a6-487d-9907-4f36349ee2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:14.238358 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1317/2647] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:14.242517 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d56bd9-406a-48e3-b926-5f0ac2d9f754 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:14.251995 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d56bd9-406a-48e3-b926-5f0ac2d9f754 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:14.257500 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9adac-1117-486c-8b47-2bd264863714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:14.258123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9adac-1117-486c-8b47-2bd264863714 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.258398 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9adac-1117-486c-8b47-2bd264863714 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.258667 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9adac-1117-486c-8b47-2bd264863714 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.259381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9adac-1117-486c-8b47-2bd264863714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:00:14.259682 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9adac-1117-486c-8b47-2bd264863714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:14.261259 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9adac-1117-486c-8b47-2bd264863714 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:14.273597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1331/2648] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 20:00:14.277695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e2edb-f2a6-487d-9907-4f36349ee2f8 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:14.287048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e2edb-f2a6-487d-9907-4f36349ee2f8 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:14.292523 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa881a-6261-4e58-ad9e-579efecdec22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:14.293628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa881a-6261-4e58-ad9e-579efecdec22 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.293863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa881a-6261-4e58-ad9e-579efecdec22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.294112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa881a-6261-4e58-ad9e-579efecdec22 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.294849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa881a-6261-4e58-ad9e-579efecdec22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:14.296742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa881a-6261-4e58-ad9e-579efecdec22 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:14.321751 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1318/2649] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:00:14 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:00:14.325849 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9adac-1117-486c-8b47-2bd264863714 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:14.336388 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9adac-1117-486c-8b47-2bd264863714 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:14.391279 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d129859-a1c7-4c3b-979d-dd9415f2e874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:14.391936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d129859-a1c7-4c3b-979d-dd9415f2e874 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.392216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d129859-a1c7-4c3b-979d-dd9415f2e874 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.392488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d129859-a1c7-4c3b-979d-dd9415f2e874 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.393179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d129859-a1c7-4c3b-979d-dd9415f2e874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:14.395722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d129859-a1c7-4c3b-979d-dd9415f2e874 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:14.415048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1332/2650] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:00:14 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:00:14.419077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa881a-6261-4e58-ad9e-579efecdec22 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:14.429845 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa881a-6261-4e58-ad9e-579efecdec22 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:14.434522 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f58eb-e0e1-413f-9f9d-da5a082c28e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:14.435171 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f58eb-e0e1-413f-9f9d-da5a082c28e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.435452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f58eb-e0e1-413f-9f9d-da5a082c28e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.435723 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f58eb-e0e1-413f-9f9d-da5a082c28e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.436443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f58eb-e0e1-413f-9f9d-da5a082c28e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:14.438309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f58eb-e0e1-413f-9f9d-da5a082c28e0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:14.449308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1319/2651] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:00:14.453200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d129859-a1c7-4c3b-979d-dd9415f2e874 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:14.463154 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d129859-a1c7-4c3b-979d-dd9415f2e874 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:14.468057 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9164d-341a-4606-b803-9515a28b81cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:14.468738 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9164d-341a-4606-b803-9515a28b81cd admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.469048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9164d-341a-4606-b803-9515a28b81cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.469323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9164d-341a-4606-b803-9515a28b81cd admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.470089 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9164d-341a-4606-b803-9515a28b81cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:14.471701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9164d-341a-4606-b803-9515a28b81cd admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:14.481719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1333/2652] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:00:14.485830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f58eb-e0e1-413f-9f9d-da5a082c28e0 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:14.496490 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f58eb-e0e1-413f-9f9d-da5a082c28e0 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:14.501174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c3915-206a-4b3a-afed-5c7c0a4aacab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:14.502081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c3915-206a-4b3a-afed-5c7c0a4aacab admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.502362 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c3915-206a-4b3a-afed-5c7c0a4aacab admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.502633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c3915-206a-4b3a-afed-5c7c0a4aacab admin admin] PATH_INFO: `/v3/projects/3d943354675349cfad1ff664576eb094/users/6ebbfd0fcf4b4a32bb801b1c9bdc99e6/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.517868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c3915-206a-4b3a-afed-5c7c0a4aacab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d943354675349cfad1ff664576eb094, user_id=6ebbfd0fcf4b4a32bb801b1c9bdc99e6, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:14.521217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c3915-206a-4b3a-afed-5c7c0a4aacab admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:14.533910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1320/2653] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:00:14 2022] PUT /identity/v3/projects/3d943354675349cfad1ff664576eb094/users/6ebbfd0fcf4b4a32bb801b1c9bdc99e6/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 20:00:14.544519 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0757bc-7010-460c-9a11-a6a64ee9a839 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.544811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0757bc-7010-460c-9a11-a6a64ee9a839 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.545088 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0757bc-7010-460c-9a11-a6a64ee9a839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.576212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-3d0757bc-7010-460c-9a11-a6a64ee9a839 None None] MFA Rules not processed for user `6ebbfd0fcf4b4a32bb801b1c9bdc99e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:14.577418 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3d0757bc-7010-460c-9a11-a6a64ee9a839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:14.646562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1334/2654] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:14.657085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c3915-206a-4b3a-afed-5c7c0a4aacab admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:14.658892 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-302c3915-206a-4b3a-afed-5c7c0a4aacab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:14.681741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:14.718806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c3915-206a-4b3a-afed-5c7c0a4aacab admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:14.722665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2566-6045-4c73-a144-e38e55845577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:14.723328 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2566-6045-4c73-a144-e38e55845577 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:14.723666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2566-6045-4c73-a144-e38e55845577 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:14.723949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2566-6045-4c73-a144-e38e55845577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:14.732645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c2566-6045-4c73-a144-e38e55845577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:14.734121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c2566-6045-4c73-a144-e38e55845577 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:14.743843 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1321/2655] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:00:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:00:18.216105 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9e98b-0bf9-404c-852a-03464f8278df None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:18.216628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9e98b-0bf9-404c-852a-03464f8278df None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:18.216628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9e98b-0bf9-404c-852a-03464f8278df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:18.234755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-b5f9e98b-0bf9-404c-852a-03464f8278df None None] MFA Rules not processed for user `6ebbfd0fcf4b4a32bb801b1c9bdc99e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:18.235931 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b5f9e98b-0bf9-404c-852a-03464f8278df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:18.244026 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1335/2656] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:24.739825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=17 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:28.229726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:28.875597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2566-6045-4c73-a144-e38e55845577 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:28.876993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9d0c2566-6045-4c73-a144-e38e55845577 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:28.888330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2566-6045-4c73-a144-e38e55845577 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:28.891885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e1b86-6c4e-4939-a26f-bda3589739c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:28.892513 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e1b86-6c4e-4939-a26f-bda3589739c8 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:28.892794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e1b86-6c4e-4939-a26f-bda3589739c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:28.893076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e1b86-6c4e-4939-a26f-bda3589739c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:28.895048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7c6e1b86-6c4e-4939-a26f-bda3589739c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:28.954168 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e1b86-6c4e-4939-a26f-bda3589739c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:28.955594 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e1b86-6c4e-4939-a26f-bda3589739c8 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:28.965058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1322/2657] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:00:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:00:37.369155 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9e98b-0bf9-404c-852a-03464f8278df None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:37.370861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b5f9e98b-0bf9-404c-852a-03464f8278df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:37.456600 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9e98b-0bf9-404c-852a-03464f8278df None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:37.462366 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:37.463474 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:37.464117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:37.464560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] PATH_INFO: `/v3/users/1255c3a12e0b44ff934d2dd19669100a` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:37.466387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1255c3a12e0b44ff934d2dd19669100a)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:37.468498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:37.528483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1255c3a12e0b44ff934d2dd19669100a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:37.541232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1255c3a12e0b44ff934d2dd19669100a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:37.541916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1255c3a12e0b44ff934d2dd19669100a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:37.562434 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1255c3a12e0b44ff934d2dd19669100a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:37.563271 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1255c3a12e0b44ff934d2dd19669100a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:37.579379 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1336/2658] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:00:37 2022] DELETE /identity/v3/users/1255c3a12e0b44ff934d2dd19669100a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:00:37.585013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356d152-2a75-4d11-812a-ccd1310f7c0a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:37.586144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356d152-2a75-4d11-812a-ccd1310f7c0a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:37.586425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356d152-2a75-4d11-812a-ccd1310f7c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:37.610615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-0356d152-2a75-4d11-812a-ccd1310f7c0a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:37.611833 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0356d152-2a75-4d11-812a-ccd1310f7c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:37.698641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1323/2659] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:37.712739 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:37.714263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:37.793212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab43655-09d7-431d-8166-3af2db39d774 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:37.799072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bde254-7e2a-43e4-b440-a20abf4eda3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:37.799817 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bde254-7e2a-43e4-b440-a20abf4eda3f service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:37.800080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bde254-7e2a-43e4-b440-a20abf4eda3f service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:37.800353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bde254-7e2a-43e4-b440-a20abf4eda3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:37.811468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bde254-7e2a-43e4-b440-a20abf4eda3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:37.812977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bde254-7e2a-43e4-b440-a20abf4eda3f service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:37.824740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1337/2660] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:00:38.029861 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d1f1f-f017-4a23-9b3b-35c4284f7c44 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:38.030532 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d1f1f-f017-4a23-9b3b-35c4284f7c44 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:38.030818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d1f1f-f017-4a23-9b3b-35c4284f7c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:38.049829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c20d1f1f-f017-4a23-9b3b-35c4284f7c44 None None] MFA Rules not processed for user `80267c31f3fc4b3fbc8dc7ed03326eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:38.051199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c20d1f1f-f017-4a23-9b3b-35c4284f7c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:38.124685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1324/2661] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:38.371642 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bde254-7e2a-43e4-b440-a20abf4eda3f service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:38.373223 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-52bde254-7e2a-43e4-b440-a20abf4eda3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:38.387179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bde254-7e2a-43e4-b440-a20abf4eda3f service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:38.391835 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:38.392702 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:38.392981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:38.393767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] PATH_INFO: `/v3/projects/4b0163c13c794461b892e18258b5d08b` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:38.394954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b0163c13c794461b892e18258b5d08b)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:38.397624 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:38.416638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:38.453145 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b0163c13c794461b892e18258b5d08b', 'request_id': 'req-272f5694-4fb5-4028-b28d-c3c9d85ad03b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:38.461377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b0163c13c794461b892e18258b5d08b', 'request_id': 'req-272f5694-4fb5-4028-b28d-c3c9d85ad03b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:38.468273 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b0163c13c794461b892e18258b5d08b', 'request_id': 'req-272f5694-4fb5-4028-b28d-c3c9d85ad03b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:38.479219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] The token cache is being invalidate because project 4b0163c13c794461b892e18258b5d08b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:00:38.479512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:38.480176 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:38.481804 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1338/2662] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:00:38 2022] DELETE /identity/v3/projects/4b0163c13c794461b892e18258b5d08b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:00:38.485956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d1f1f-f017-4a23-9b3b-35c4284f7c44 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:38.498469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d1f1f-f017-4a23-9b3b-35c4284f7c44 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:38.555203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:38.584856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:38.586279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:38.586279 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:38.586965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] PATH_INFO: `/v3/users/b71c9e74549e4745ace480ea0b5adf30` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:38.587726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b71c9e74549e4745ace480ea0b5adf30)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:38.589687 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:38.652582 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b71c9e74549e4745ace480ea0b5adf30'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:38.668151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b71c9e74549e4745ace480ea0b5adf30'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:38.678105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b71c9e74549e4745ace480ea0b5adf30'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:38.687995 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b71c9e74549e4745ace480ea0b5adf30'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:38.689073 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-aeb5d465-001d-4f36-b5f9-36a308f99d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b71c9e74549e4745ace480ea0b5adf30'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:38.701770 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1325/2663] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:00:38 2022] DELETE /identity/v3/users/b71c9e74549e4745ace480ea0b5adf30 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 20:00:39.205818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:39.207575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:39.236266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f5694-4fb5-4028-b28d-c3c9d85ad03b admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:39.276597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0a31e-c5fd-40e9-be9f-acaa27dcf2f2 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:39.277144 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0a31e-c5fd-40e9-be9f-acaa27dcf2f2 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:39.277676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0a31e-c5fd-40e9-be9f-acaa27dcf2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:39.331633 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-0ca0a31e-c5fd-40e9-be9f-acaa27dcf2f2 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:39.333972 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0ca0a31e-c5fd-40e9-be9f-acaa27dcf2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:39.360296 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:39.361277 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:39.361554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:39.361898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] PATH_INFO: `/v3/projects/dd5a72da634c45828b6983d093e23b4c` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:39.363344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd5a72da634c45828b6983d093e23b4c)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:39.365813 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:39.425403 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1326/2664] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 20:00:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:39.460267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd5a72da634c45828b6983d093e23b4c', 'request_id': 'req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:39.476408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd5a72da634c45828b6983d093e23b4c', 'request_id': 'req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:39.488296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0a31e-c5fd-40e9-be9f-acaa27dcf2f2 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:39.488828 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd5a72da634c45828b6983d093e23b4c', 'request_id': 'req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:39.491629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0ca0a31e-c5fd-40e9-be9f-acaa27dcf2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:39.513427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] The token cache is being invalidate because project dd5a72da634c45828b6983d093e23b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:00:39.513742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:39.516697 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-b0be5330-bb4a-46c7-8370-1eb7c29bd3f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:39.519157 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1339/2665] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:00:39 2022] DELETE /identity/v3/projects/dd5a72da634c45828b6983d093e23b4c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:00:39.525962 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939502-e75a-43b0-9a4e-cfa45ef9365d None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:39.526686 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939502-e75a-43b0-9a4e-cfa45ef9365d None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:39.527333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939502-e75a-43b0-9a4e-cfa45ef9365d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:39.551387 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-15939502-e75a-43b0-9a4e-cfa45ef9365d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:39.553509 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-15939502-e75a-43b0-9a4e-cfa45ef9365d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:39.594543 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0a31e-c5fd-40e9-be9f-acaa27dcf2f2 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:39.595083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:39.611731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:39.653860 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1340/2666] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:39.655344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113c462-f299-4ca4-8582-63b46c4dde4e None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:39.655634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113c462-f299-4ca4-8582-63b46c4dde4e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:39.655911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113c462-f299-4ca4-8582-63b46c4dde4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:39.677790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5113c462-f299-4ca4-8582-63b46c4dde4e None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:39.678864 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5113c462-f299-4ca4-8582-63b46c4dde4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:39.689706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b5bde-d792-4b7d-b1ef-56914b29bbc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:39.689952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1341/2667] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:39.690829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b5bde-d792-4b7d-b1ef-56914b29bbc5 service swift] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:39.691181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b5bde-d792-4b7d-b1ef-56914b29bbc5 service swift] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:39.691438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113c462-f299-4ca4-8582-63b46c4dde4e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:39.691546 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b5bde-d792-4b7d-b1ef-56914b29bbc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:39.693549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b61b5bde-d792-4b7d-b1ef-56914b29bbc5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:39.702529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113c462-f299-4ca4-8582-63b46c4dde4e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:39.708633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560fee8-a7d8-49f8-9c36-ffac4d9632b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:39.709552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560fee8-a7d8-49f8-9c36-ffac4d9632b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:39.709552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560fee8-a7d8-49f8-9c36-ffac4d9632b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:39.709923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560fee8-a7d8-49f8-9c36-ffac4d9632b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:39.710623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560fee8-a7d8-49f8-9c36-ffac4d9632b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:00:39.710907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560fee8-a7d8-49f8-9c36-ffac4d9632b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:39.712522 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560fee8-a7d8-49f8-9c36-ffac4d9632b1 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:39.728439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1342/2668] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 20:00:39.734172 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560fee8-a7d8-49f8-9c36-ffac4d9632b1 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:39.746217 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560fee8-a7d8-49f8-9c36-ffac4d9632b1 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:39.753336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae0afb-bcd1-4799-9ec2-530e9adf2c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:39.754313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae0afb-bcd1-4799-9ec2-530e9adf2c6c admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:39.755012 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae0afb-bcd1-4799-9ec2-530e9adf2c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:39.755433 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae0afb-bcd1-4799-9ec2-530e9adf2c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:39.756246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae0afb-bcd1-4799-9ec2-530e9adf2c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:39.758729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae0afb-bcd1-4799-9ec2-530e9adf2c6c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:39.796445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b5bde-d792-4b7d-b1ef-56914b29bbc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:39.798320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b5bde-d792-4b7d-b1ef-56914b29bbc5 service swift] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:39.808230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1343/2669] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:00:39 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:00:39.808631 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1327/2670] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 20:00:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 20:00:39.812063 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae0afb-bcd1-4799-9ec2-530e9adf2c6c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:39.821613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac42725-f87f-4a8d-89f3-3239e29c12f6 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:39.821968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac42725-f87f-4a8d-89f3-3239e29c12f6 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:39.822423 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac42725-f87f-4a8d-89f3-3239e29c12f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:39.845935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-3ac42725-f87f-4a8d-89f3-3239e29c12f6 None None] MFA Rules not processed for user `8f2b9a6934d34f58a09a4cefb154371c`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:39.847434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-3ac42725-f87f-4a8d-89f3-3239e29c12f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:39.903232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae0afb-bcd1-4799-9ec2-530e9adf2c6c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:39.909047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acecfea-11d6-4d7b-a888-226490995b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:39.909869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acecfea-11d6-4d7b-a888-226490995b22 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:39.910254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acecfea-11d6-4d7b-a888-226490995b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:39.910597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acecfea-11d6-4d7b-a888-226490995b22 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:39.912949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acecfea-11d6-4d7b-a888-226490995b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:39.915188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acecfea-11d6-4d7b-a888-226490995b22 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:39.941548 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1344/2671] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:00:39 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:00:39.947336 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1328/2672] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:39.947530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acecfea-11d6-4d7b-a888-226490995b22 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:39.956818 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acecfea-11d6-4d7b-a888-226490995b22 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:39.963710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2de2a-cc20-4357-9cb4-1dc76e21e2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:39.964693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2de2a-cc20-4357-9cb4-1dc76e21e2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:39.965073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2de2a-cc20-4357-9cb4-1dc76e21e2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:39.965413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2de2a-cc20-4357-9cb4-1dc76e21e2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:39.966504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2de2a-cc20-4357-9cb4-1dc76e21e2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:39.968545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2de2a-cc20-4357-9cb4-1dc76e21e2b9 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:39.980578 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1345/2673] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:00:39.984680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac42725-f87f-4a8d-89f3-3239e29c12f6 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:39.997071 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac42725-f87f-4a8d-89f3-3239e29c12f6 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:40.002185 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fdc24-0585-4ba3-bca7-95d15bb70b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:40.002856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fdc24-0585-4ba3-bca7-95d15bb70b93 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.003128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fdc24-0585-4ba3-bca7-95d15bb70b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.003404 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fdc24-0585-4ba3-bca7-95d15bb70b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.004149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fdc24-0585-4ba3-bca7-95d15bb70b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:40.006739 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fdc24-0585-4ba3-bca7-95d15bb70b93 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:40.016539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1329/2674] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:00:40.021318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2de2a-cc20-4357-9cb4-1dc76e21e2b9 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:40.031171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2de2a-cc20-4357-9cb4-1dc76e21e2b9 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:40.037797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451dab1f-436d-4a24-8b74-b7192b44a86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:40.038698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dab1f-436d-4a24-8b74-b7192b44a86f admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.038976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dab1f-436d-4a24-8b74-b7192b44a86f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.039246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dab1f-436d-4a24-8b74-b7192b44a86f admin admin] PATH_INFO: `/v3/projects/74d4ad2e27b64084ad0cefaed213d509/users/1203f74f500d4597a18e8ea6b7bd0fae/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.057678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dab1f-436d-4a24-8b74-b7192b44a86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d4ad2e27b64084ad0cefaed213d509, user_id=1203f74f500d4597a18e8ea6b7bd0fae, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:40.060351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dab1f-436d-4a24-8b74-b7192b44a86f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:40.075103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1346/2675] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:00:40 2022] PUT /identity/v3/projects/74d4ad2e27b64084ad0cefaed213d509/users/1203f74f500d4597a18e8ea6b7bd0fae/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:00:40.086618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d880d-0192-46f4-9bd3-dbb793743351 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.086910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d880d-0192-46f4-9bd3-dbb793743351 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.087189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d880d-0192-46f4-9bd3-dbb793743351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.120975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d58d880d-0192-46f4-9bd3-dbb793743351 None None] MFA Rules not processed for user `1203f74f500d4597a18e8ea6b7bd0fae`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:40.122372 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d58d880d-0192-46f4-9bd3-dbb793743351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:40.174938 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:40.229057 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1330/2676] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:40.232890 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451dab1f-436d-4a24-8b74-b7192b44a86f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:40.320616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:40.350235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451dab1f-436d-4a24-8b74-b7192b44a86f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:40.380833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030cc12-5756-4cf4-abda-a2338d36e864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:40.381793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030cc12-5756-4cf4-abda-a2338d36e864 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.382202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030cc12-5756-4cf4-abda-a2338d36e864 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.382752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030cc12-5756-4cf4-abda-a2338d36e864 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.383537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030cc12-5756-4cf4-abda-a2338d36e864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:40.385766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030cc12-5756-4cf4-abda-a2338d36e864 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:40.433863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d880d-0192-46f4-9bd3-dbb793743351 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:40.437866 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1347/2677] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:00:40 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:00:40.454434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d880d-0192-46f4-9bd3-dbb793743351 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:40.532417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952166e-51f5-4e6d-b1b0-d4e5a7195cd5 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.533228 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952166e-51f5-4e6d-b1b0-d4e5a7195cd5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.533740 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952166e-51f5-4e6d-b1b0-d4e5a7195cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.554389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-e952166e-51f5-4e6d-b1b0-d4e5a7195cd5 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:40.555604 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e952166e-51f5-4e6d-b1b0-d4e5a7195cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:40.558969 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49ff13-5b52-4940-abbc-d0736d8d12c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:40.559796 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49ff13-5b52-4940-abbc-d0736d8d12c3 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.560176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49ff13-5b52-4940-abbc-d0736d8d12c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.560664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49ff13-5b52-4940-abbc-d0736d8d12c3 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.562080 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49ff13-5b52-4940-abbc-d0736d8d12c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:40.563889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49ff13-5b52-4940-abbc-d0736d8d12c3 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:40.589364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1331/2678] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:00:40 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:00:40.593573 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49ff13-5b52-4940-abbc-d0736d8d12c3 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:40.604772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49ff13-5b52-4940-abbc-d0736d8d12c3 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:40.611047 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e71f4-0c41-435d-928d-63d3709eadfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:40.612026 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e71f4-0c41-435d-928d-63d3709eadfc admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.612147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e71f4-0c41-435d-928d-63d3709eadfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.612381 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e71f4-0c41-435d-928d-63d3709eadfc admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.613108 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e71f4-0c41-435d-928d-63d3709eadfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:40.616436 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e71f4-0c41-435d-928d-63d3709eadfc admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:40.623911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1348/2679] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 1 20:00:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 20:00:40.627237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1332/2680] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 20:00:40.638173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952166e-51f5-4e6d-b1b0-d4e5a7195cd5 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:40.643289 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e71f4-0c41-435d-928d-63d3709eadfc admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:40.645299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-ce6e71f4-0c41-435d-928d-63d3709eadfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:40.652639 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952166e-51f5-4e6d-b1b0-d4e5a7195cd5 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:40.659370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6a2e3-396e-485b-a7ff-6a3f328a9153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:40.660331 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6a2e3-396e-485b-a7ff-6a3f328a9153 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.660616 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6a2e3-396e-485b-a7ff-6a3f328a9153 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.661414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6a2e3-396e-485b-a7ff-6a3f328a9153 admin admin] PATH_INFO: `/v3/projects/dc42ba341201446caf1ce4d889bff321/users/a72740d938604a79956331c567f289e7/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.681376 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6a2e3-396e-485b-a7ff-6a3f328a9153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc42ba341201446caf1ce4d889bff321, user_id=a72740d938604a79956331c567f289e7, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:40.684881 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6a2e3-396e-485b-a7ff-6a3f328a9153 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:40.702654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1349/2681] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:00:40 2022] PUT /identity/v3/projects/dc42ba341201446caf1ce4d889bff321/users/a72740d938604a79956331c567f289e7/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:00:40.707010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6a2e3-396e-485b-a7ff-6a3f328a9153 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:40.731054 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e71f4-0c41-435d-928d-63d3709eadfc admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:40.792694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:40.793693 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6a2e3-396e-485b-a7ff-6a3f328a9153 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:40.802597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31754bd0-b8ec-461e-9a1c-b3b69d782691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:40.803907 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31754bd0-b8ec-461e-9a1c-b3b69d782691 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.804566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31754bd0-b8ec-461e-9a1c-b3b69d782691 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.804827 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31754bd0-b8ec-461e-9a1c-b3b69d782691 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.806851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31754bd0-b8ec-461e-9a1c-b3b69d782691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:40.810079 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31754bd0-b8ec-461e-9a1c-b3b69d782691 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:40.811876 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a650ff-c130-4fb7-a515-ce9efab31120 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:40.812400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a650ff-c130-4fb7-a515-ce9efab31120 service swift] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.812777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a650ff-c130-4fb7-a515-ce9efab31120 service swift] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.813063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a650ff-c130-4fb7-a515-ce9efab31120 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.857825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1350/2682] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:00:40.857825 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31754bd0-b8ec-461e-9a1c-b3b69d782691 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:40.871454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31754bd0-b8ec-461e-9a1c-b3b69d782691 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:40.878214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e562b5-9edc-4868-92c7-3d9a5360185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:40.879372 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e562b5-9edc-4868-92c7-3d9a5360185c admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.880780 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e562b5-9edc-4868-92c7-3d9a5360185c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.881152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e562b5-9edc-4868-92c7-3d9a5360185c admin admin] PATH_INFO: `/v3/projects/dc42ba341201446caf1ce4d889bff321/users/a72740d938604a79956331c567f289e7/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.884184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e562b5-9edc-4868-92c7-3d9a5360185c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc42ba341201446caf1ce4d889bff321, user_id=a72740d938604a79956331c567f289e7, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:40.887091 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e562b5-9edc-4868-92c7-3d9a5360185c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:40.914477 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1351/2683] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:00:40 2022] PUT /identity/v3/projects/dc42ba341201446caf1ce4d889bff321/users/a72740d938604a79956331c567f289e7/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:00:40.915730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bb81b-e939-4f39-aec9-7276d2ad0f2c None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:40.915871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bb81b-e939-4f39-aec9-7276d2ad0f2c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:40.916107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bb81b-e939-4f39-aec9-7276d2ad0f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:40.947640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a650ff-c130-4fb7-a515-ce9efab31120 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:40.950079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a650ff-c130-4fb7-a515-ce9efab31120 service swift] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:40.955812 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a67bb81b-e939-4f39-aec9-7276d2ad0f2c None None] MFA Rules not processed for user `a72740d938604a79956331c567f289e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:40.957435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a67bb81b-e939-4f39-aec9-7276d2ad0f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:40.965514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1333/2684] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 20:00:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 20:00:41.020293 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:41.109841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1352/2685] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:41.122425 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a650ff-c130-4fb7-a515-ce9efab31120 service swift] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:41.128107 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-79a650ff-c130-4fb7-a515-ce9efab31120 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:41.232957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a650ff-c130-4fb7-a515-ce9efab31120 service swift] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:41.264734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df298-ae82-482e-8555-aa43570e03ac None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:41.265224 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df298-ae82-482e-8555-aa43570e03ac None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:41.266076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df298-ae82-482e-8555-aa43570e03ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:41.292709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-930df298-ae82-482e-8555-aa43570e03ac None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:41.294199 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-930df298-ae82-482e-8555-aa43570e03ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:41.314012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc02b7-9929-4793-bba1-0c034c14485a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:41.314661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc02b7-9929-4793-bba1-0c034c14485a service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:41.315293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc02b7-9929-4793-bba1-0c034c14485a service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:41.315293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc02b7-9929-4793-bba1-0c034c14485a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:41.382265 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1353/2686] 10.222.0.66 () {58 vars in 1009 bytes} [Fri Jul 1 20:00:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:41.397862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930df298-ae82-482e-8555-aa43570e03ac None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:41.423036 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc02b7-9929-4793-bba1-0c034c14485a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:41.425254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc02b7-9929-4793-bba1-0c034c14485a service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:41.436359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1334/2687] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:00:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:00:41.500574 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930df298-ae82-482e-8555-aa43570e03ac None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:41.515290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace12c55-e4b3-4acb-8a76-69932830f8fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:41.516567 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace12c55-e4b3-4acb-8a76-69932830f8fb service swift] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:41.516976 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace12c55-e4b3-4acb-8a76-69932830f8fb service swift] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:41.517314 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace12c55-e4b3-4acb-8a76-69932830f8fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:41.541834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace12c55-e4b3-4acb-8a76-69932830f8fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:41.543572 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace12c55-e4b3-4acb-8a76-69932830f8fb service swift] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:41.585095 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1354/2688] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 20:00:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 20:00:42.051189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c2a41-8c53-4619-84d7-c9f167e77780 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:42.051829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c2a41-8c53-4619-84d7-c9f167e77780 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:42.051829 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c2a41-8c53-4619-84d7-c9f167e77780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:42.077445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-377c2a41-8c53-4619-84d7-c9f167e77780 None None] MFA Rules not processed for user `cfa25a63dd6f4ad08b0f858356528284`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:42.078865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-377c2a41-8c53-4619-84d7-c9f167e77780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:42.088087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1335/2689] 10.222.0.66 () {58 vars in 967 bytes} [Fri Jul 1 20:00:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:42.106765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace12c55-e4b3-4acb-8a76-69932830f8fb service swift] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:42.119977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace12c55-e4b3-4acb-8a76-69932830f8fb service swift] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:42.125278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ea459-2002-4792-b34c-fa3581a0e5ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2bce9daec8c446d89eb6cc5950d6649', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:42.125987 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ea459-2002-4792-b34c-fa3581a0e5ad service swift] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:42.126342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ea459-2002-4792-b34c-fa3581a0e5ad service swift] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:42.126342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ea459-2002-4792-b34c-fa3581a0e5ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:42.137843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ea459-2002-4792-b34c-fa3581a0e5ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:42.140629 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ea459-2002-4792-b34c-fa3581a0e5ad service swift] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:42.156066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1355/2690] 10.222.0.66 () {60 vars in 1384 bytes} [Fri Jul 1 20:00:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 20:00:47.547114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c2a41-8c53-4619-84d7-c9f167e77780 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:47.553134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:47.559545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c2a41-8c53-4619-84d7-c9f167e77780 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:47.563146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a149e-7654-4135-b99b-50d8ce70c434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:47.563787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a149e-7654-4135-b99b-50d8ce70c434 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:47.564155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a149e-7654-4135-b99b-50d8ce70c434 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:47.564434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a149e-7654-4135-b99b-50d8ce70c434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:47.574501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a149e-7654-4135-b99b-50d8ce70c434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:47.576042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a149e-7654-4135-b99b-50d8ce70c434 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:47.589241 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1336/2691] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:00:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:00:47.644879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ea459-2002-4792-b34c-fa3581a0e5ad service swift] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:47.647047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4d8ea459-2002-4792-b34c-fa3581a0e5ad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:47.744773 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=100 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:47.787431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ea459-2002-4792-b34c-fa3581a0e5ad service swift] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:47.793344 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:47.794557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:47.794902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:47.795243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] PATH_INFO: `/v3/users/6ebbfd0fcf4b4a32bb801b1c9bdc99e6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:47.796832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ebbfd0fcf4b4a32bb801b1c9bdc99e6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:47.799060 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:47.879595 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ebbfd0fcf4b4a32bb801b1c9bdc99e6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:47.888936 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ebbfd0fcf4b4a32bb801b1c9bdc99e6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:47.889779 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ebbfd0fcf4b4a32bb801b1c9bdc99e6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:47.904910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ebbfd0fcf4b4a32bb801b1c9bdc99e6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:47.905662 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ebbfd0fcf4b4a32bb801b1c9bdc99e6'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:47.919895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1356/2692] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:00:47 2022] DELETE /identity/v3/users/6ebbfd0fcf4b4a32bb801b1c9bdc99e6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:00:47.924141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b9853-d09e-4520-8799-c8ea014326aa None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:47.924433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b9853-d09e-4520-8799-c8ea014326aa None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:47.924711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b9853-d09e-4520-8799-c8ea014326aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:47.943998 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-5d3b9853-d09e-4520-8799-c8ea014326aa None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:00:47.945220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5d3b9853-d09e-4520-8799-c8ea014326aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:48.059942 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1337/2693] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:00:48.069190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:48.070880 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:48.135916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84683f60-a7ab-4997-9e25-709415c7141f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:48.139447 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa7a0c-931c-4345-a08a-13f2af81eeb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:48.140159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa7a0c-931c-4345-a08a-13f2af81eeb3 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:48.140501 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa7a0c-931c-4345-a08a-13f2af81eeb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:48.140894 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa7a0c-931c-4345-a08a-13f2af81eeb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:48.151843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa7a0c-931c-4345-a08a-13f2af81eeb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:48.153306 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa7a0c-931c-4345-a08a-13f2af81eeb3 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:48.164059 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1357/2694] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 20:00:48.353456 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b9853-d09e-4520-8799-c8ea014326aa None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:48.355067 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5d3b9853-d09e-4520-8799-c8ea014326aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:48.371558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b9853-d09e-4520-8799-c8ea014326aa None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:48.376904 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:48.377799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:48.378077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:48.378386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] PATH_INFO: `/v3/projects/3d943354675349cfad1ff664576eb094` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:48.381277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d943354675349cfad1ff664576eb094)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:48.383900 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:48.433575 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d943354675349cfad1ff664576eb094', 'request_id': 'req-cf3bb880-7805-4a81-b930-28d6aa766910'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:48.441455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d943354675349cfad1ff664576eb094', 'request_id': 'req-cf3bb880-7805-4a81-b930-28d6aa766910'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:48.447657 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d943354675349cfad1ff664576eb094', 'request_id': 'req-cf3bb880-7805-4a81-b930-28d6aa766910'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:48.459420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] The token cache is being invalidate because project 3d943354675349cfad1ff664576eb094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:00:48.459703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:48.460331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:00:48.461891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1338/2695] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:00:48 2022] DELETE /identity/v3/projects/3d943354675349cfad1ff664576eb094 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:00:56.698526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa7a0c-931c-4345-a08a-13f2af81eeb3 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:00:56.699934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-84aa7a0c-931c-4345-a08a-13f2af81eeb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:56.776025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa7a0c-931c-4345-a08a-13f2af81eeb3 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:00:56.780457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:00:56.837216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790c74d-af99-4716-98ed-2cdc69d960cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:00:56.837900 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790c74d-af99-4716-98ed-2cdc69d960cb service placement] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:00:56.838183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790c74d-af99-4716-98ed-2cdc69d960cb service placement] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:00:56.838456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790c74d-af99-4716-98ed-2cdc69d960cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:00:56.840255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5790c74d-af99-4716-98ed-2cdc69d960cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:00:56.852081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790c74d-af99-4716-98ed-2cdc69d960cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:00:56.853767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790c74d-af99-4716-98ed-2cdc69d960cb service placement] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:00:56.865504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1358/2696] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:00:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:00:58.455766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:06.859654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=28 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:12.633726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:12.635149 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:12.698447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bb880-7805-4a81-b930-28d6aa766910 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:12.702994 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:12.703860 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:12.704141 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:12.704417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] PATH_INFO: `/v3/users/1203f74f500d4597a18e8ea6b7bd0fae` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:12.705620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1203f74f500d4597a18e8ea6b7bd0fae)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:12.707587 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:12.753230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1203f74f500d4597a18e8ea6b7bd0fae'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:12.766269 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1203f74f500d4597a18e8ea6b7bd0fae'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:12.773662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1203f74f500d4597a18e8ea6b7bd0fae'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:12.781439 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1203f74f500d4597a18e8ea6b7bd0fae'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:12.782100 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1203f74f500d4597a18e8ea6b7bd0fae'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:12.787951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1339/2697] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:01:12 2022] DELETE /identity/v3/users/1203f74f500d4597a18e8ea6b7bd0fae => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:12.792443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed992b-69b4-46a9-9259-64d47c74132f None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:12.792920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed992b-69b4-46a9-9259-64d47c74132f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:12.793036 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed992b-69b4-46a9-9259-64d47c74132f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:12.810720 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-49ed992b-69b4-46a9-9259-64d47c74132f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:12.811923 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-49ed992b-69b4-46a9-9259-64d47c74132f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:12.875215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1359/2698] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:12.884671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:12.886094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:12.946061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd47b0-0490-4d4a-9f45-22dc3ce3a98c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:12.949775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a5ca2-a514-41a9-a00f-87ffa52f390e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:12.950422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a5ca2-a514-41a9-a00f-87ffa52f390e service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:12.950706 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a5ca2-a514-41a9-a00f-87ffa52f390e service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:12.950986 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a5ca2-a514-41a9-a00f-87ffa52f390e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:12.960705 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a5ca2-a514-41a9-a00f-87ffa52f390e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:12.962176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a5ca2-a514-41a9-a00f-87ffa52f390e service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:12.971645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1340/2699] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:01:13.244647 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed992b-69b4-46a9-9259-64d47c74132f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:13.246103 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-49ed992b-69b4-46a9-9259-64d47c74132f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:13.259464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed992b-69b4-46a9-9259-64d47c74132f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:13.264837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:13.265772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:13.266044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:13.266317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] PATH_INFO: `/v3/projects/74d4ad2e27b64084ad0cefaed213d509` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:13.268162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d4ad2e27b64084ad0cefaed213d509)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:13.271587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:13.320457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d4ad2e27b64084ad0cefaed213d509', 'request_id': 'req-abcc2929-d86c-4248-8df1-17bef487997f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.328683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d4ad2e27b64084ad0cefaed213d509', 'request_id': 'req-abcc2929-d86c-4248-8df1-17bef487997f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.335951 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d4ad2e27b64084ad0cefaed213d509', 'request_id': 'req-abcc2929-d86c-4248-8df1-17bef487997f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.346097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] The token cache is being invalidate because project 74d4ad2e27b64084ad0cefaed213d509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:01:13.346380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.347003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.349586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1360/2700] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:01:13 2022] DELETE /identity/v3/projects/74d4ad2e27b64084ad0cefaed213d509 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:13.353624 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a5ca2-a514-41a9-a00f-87ffa52f390e service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:13.363138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a5ca2-a514-41a9-a00f-87ffa52f390e service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:13.381203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:13.422000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:13.422868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:13.423152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:13.423429 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] PATH_INFO: `/v3/users/a72740d938604a79956331c567f289e7` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:13.424612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a72740d938604a79956331c567f289e7)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:13.426584 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:13.473242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a72740d938604a79956331c567f289e7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.484855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a72740d938604a79956331c567f289e7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.491946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a72740d938604a79956331c567f289e7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.500630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a72740d938604a79956331c567f289e7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.501302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f7862317-a53a-4a85-a24a-795d93b47533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a72740d938604a79956331c567f289e7'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.507409 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1341/2701] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:01:13 2022] DELETE /identity/v3/users/a72740d938604a79956331c567f289e7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 20:01:13.779638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:13.781162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:13.794811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc2929-d86c-4248-8df1-17bef487997f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:13.857997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:13.858774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:13.859051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:13.859320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] PATH_INFO: `/v3/projects/dc42ba341201446caf1ce4d889bff321` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:13.860584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc42ba341201446caf1ce4d889bff321)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:13.862589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:13.896747 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:13.914608 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc42ba341201446caf1ce4d889bff321', 'request_id': 'req-29cd9474-df45-4ff0-a004-26405503dce9'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.922324 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc42ba341201446caf1ce4d889bff321', 'request_id': 'req-29cd9474-df45-4ff0-a004-26405503dce9'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.929082 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc42ba341201446caf1ce4d889bff321', 'request_id': 'req-29cd9474-df45-4ff0-a004-26405503dce9'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.939278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] The token cache is being invalidate because project dc42ba341201446caf1ce4d889bff321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:01:13.939564 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.940200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-29cd9474-df45-4ff0-a004-26405503dce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:13.942478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1361/2702] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:01:13 2022] DELETE /identity/v3/projects/dc42ba341201446caf1ce4d889bff321 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:13.947607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c16cd8-5e84-4317-954c-9acc81e8cf1d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:13.948105 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c16cd8-5e84-4317-954c-9acc81e8cf1d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:13.948203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c16cd8-5e84-4317-954c-9acc81e8cf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:13.966537 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d6c16cd8-5e84-4317-954c-9acc81e8cf1d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:13.967750 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d6c16cd8-5e84-4317-954c-9acc81e8cf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:14.026799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1342/2703] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:14.036173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01191-9507-40b1-9065-c93879bd0bb0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.036521 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01191-9507-40b1-9065-c93879bd0bb0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.036805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01191-9507-40b1-9065-c93879bd0bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.055527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-81b01191-9507-40b1-9065-c93879bd0bb0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:14.056910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-81b01191-9507-40b1-9065-c93879bd0bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:14.064774 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1362/2704] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:14.069795 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c16cd8-5e84-4317-954c-9acc81e8cf1d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.079493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c16cd8-5e84-4317-954c-9acc81e8cf1d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.084023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b779ea-9d95-49a6-9a67-ddf30bd2e48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.084674 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b779ea-9d95-49a6-9a67-ddf30bd2e48b admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.084956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b779ea-9d95-49a6-9a67-ddf30bd2e48b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.085230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b779ea-9d95-49a6-9a67-ddf30bd2e48b admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.085984 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b779ea-9d95-49a6-9a67-ddf30bd2e48b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:01:14.086291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b779ea-9d95-49a6-9a67-ddf30bd2e48b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.087909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b779ea-9d95-49a6-9a67-ddf30bd2e48b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.099430 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1343/2705] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 20:01:14.103615 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b01191-9507-40b1-9065-c93879bd0bb0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.113318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b01191-9507-40b1-9065-c93879bd0bb0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.117954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f256d3-9c15-464b-b20f-6632beecd655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.118594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f256d3-9c15-464b-b20f-6632beecd655 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.118871 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f256d3-9c15-464b-b20f-6632beecd655 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.119143 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f256d3-9c15-464b-b20f-6632beecd655 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.119828 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f256d3-9c15-464b-b20f-6632beecd655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.121777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f256d3-9c15-464b-b20f-6632beecd655 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.148508 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1363/2706] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:01:14 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:01:14.152779 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b779ea-9d95-49a6-9a67-ddf30bd2e48b admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.163228 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b779ea-9d95-49a6-9a67-ddf30bd2e48b admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.195253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:14.228716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce36065-c7bb-43df-8437-a66afb5295a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.229387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce36065-c7bb-43df-8437-a66afb5295a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.229710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce36065-c7bb-43df-8437-a66afb5295a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.229991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce36065-c7bb-43df-8437-a66afb5295a0 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.230682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce36065-c7bb-43df-8437-a66afb5295a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.232620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce36065-c7bb-43df-8437-a66afb5295a0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.252484 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1344/2707] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:01:14 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:01:14.257385 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f256d3-9c15-464b-b20f-6632beecd655 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.267892 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f256d3-9c15-464b-b20f-6632beecd655 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.272756 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156149c-bcc9-4e52-bcd5-bfd1796a6e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.273607 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156149c-bcc9-4e52-bcd5-bfd1796a6e1a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.273902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156149c-bcc9-4e52-bcd5-bfd1796a6e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.274178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156149c-bcc9-4e52-bcd5-bfd1796a6e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.274906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156149c-bcc9-4e52-bcd5-bfd1796a6e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.276504 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156149c-bcc9-4e52-bcd5-bfd1796a6e1a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.286898 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1364/2708] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:01:14.290801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce36065-c7bb-43df-8437-a66afb5295a0 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.301227 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce36065-c7bb-43df-8437-a66afb5295a0 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.306210 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b9f9b-0941-42cc-b12a-a1c6c7c857a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.306934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b9f9b-0941-42cc-b12a-a1c6c7c857a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.307226 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b9f9b-0941-42cc-b12a-a1c6c7c857a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.307506 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b9f9b-0941-42cc-b12a-a1c6c7c857a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.308235 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b9f9b-0941-42cc-b12a-a1c6c7c857a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.309884 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b9f9b-0941-42cc-b12a-a1c6c7c857a6 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.320764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1345/2709] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:01:14.324893 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156149c-bcc9-4e52-bcd5-bfd1796a6e1a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.334992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156149c-bcc9-4e52-bcd5-bfd1796a6e1a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.339641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c1304-bf91-475c-a5fd-bb12df6be340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.340529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c1304-bf91-475c-a5fd-bb12df6be340 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.340810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c1304-bf91-475c-a5fd-bb12df6be340 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.341086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c1304-bf91-475c-a5fd-bb12df6be340 admin admin] PATH_INFO: `/v3/projects/2c4f04845f1b4987b29cafc3971c75d6/users/dffbeae0c97640299fa6366c45658cc1/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.355986 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c1304-bf91-475c-a5fd-bb12df6be340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4f04845f1b4987b29cafc3971c75d6, user_id=dffbeae0c97640299fa6366c45658cc1, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.358675 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c1304-bf91-475c-a5fd-bb12df6be340 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.372119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1365/2710] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:01:14 2022] PUT /identity/v3/projects/2c4f04845f1b4987b29cafc3971c75d6/users/dffbeae0c97640299fa6366c45658cc1/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:14.382450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f77d0-5a69-4225-bed0-820b5006ed3a None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.382764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f77d0-5a69-4225-bed0-820b5006ed3a None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.383042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f77d0-5a69-4225-bed0-820b5006ed3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.409251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7d9f77d0-5a69-4225-bed0-820b5006ed3a None None] MFA Rules not processed for user `dffbeae0c97640299fa6366c45658cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:14.411021 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7d9f77d0-5a69-4225-bed0-820b5006ed3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:14.474204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1346/2711] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:14.478654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c1304-bf91-475c-a5fd-bb12df6be340 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.543883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c1304-bf91-475c-a5fd-bb12df6be340 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.548673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39802a-7e8d-4dc6-a923-466d3a2c8b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.549441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39802a-7e8d-4dc6-a923-466d3a2c8b90 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.549832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39802a-7e8d-4dc6-a923-466d3a2c8b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.550178 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39802a-7e8d-4dc6-a923-466d3a2c8b90 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.551627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39802a-7e8d-4dc6-a923-466d3a2c8b90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.553708 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39802a-7e8d-4dc6-a923-466d3a2c8b90 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.579188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1366/2712] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:01:14 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:01:14.583360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f77d0-5a69-4225-bed0-820b5006ed3a None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.594703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f77d0-5a69-4225-bed0-820b5006ed3a None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.651460 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e2446-a4cc-4018-9daa-3c50fc249cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.652046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e2446-a4cc-4018-9daa-3c50fc249cab admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.652327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e2446-a4cc-4018-9daa-3c50fc249cab admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.652602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e2446-a4cc-4018-9daa-3c50fc249cab admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.653291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e2446-a4cc-4018-9daa-3c50fc249cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.655234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e2446-a4cc-4018-9daa-3c50fc249cab admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.679212 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1347/2713] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:01:14 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:01:14.680450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39802a-7e8d-4dc6-a923-466d3a2c8b90 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.690623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39802a-7e8d-4dc6-a923-466d3a2c8b90 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.695542 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e656a08-461c-47b7-b02b-5ff0cd37b507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.696195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e656a08-461c-47b7-b02b-5ff0cd37b507 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.696476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e656a08-461c-47b7-b02b-5ff0cd37b507 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.696749 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e656a08-461c-47b7-b02b-5ff0cd37b507 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.697468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e656a08-461c-47b7-b02b-5ff0cd37b507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.699126 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e656a08-461c-47b7-b02b-5ff0cd37b507 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.709445 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1367/2714] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:01:14.713302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e2446-a4cc-4018-9daa-3c50fc249cab admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.724014 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e2446-a4cc-4018-9daa-3c50fc249cab admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.728593 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab44cb-953a-4137-8f11-d3ace1c5fe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.729462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab44cb-953a-4137-8f11-d3ace1c5fe6a admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.729774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab44cb-953a-4137-8f11-d3ace1c5fe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.730049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab44cb-953a-4137-8f11-d3ace1c5fe6a admin admin] PATH_INFO: `/v3/projects/8bc2bd402ee446b19bb05c41d61bc2af/users/6ff1126dfc014e91aa3535769ce44c3a/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.745294 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab44cb-953a-4137-8f11-d3ace1c5fe6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc2bd402ee446b19bb05c41d61bc2af, user_id=6ff1126dfc014e91aa3535769ce44c3a, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.747951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab44cb-953a-4137-8f11-d3ace1c5fe6a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.760664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1348/2715] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:01:14 2022] PUT /identity/v3/projects/8bc2bd402ee446b19bb05c41d61bc2af/users/6ff1126dfc014e91aa3535769ce44c3a/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:14.764619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e656a08-461c-47b7-b02b-5ff0cd37b507 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.775459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e656a08-461c-47b7-b02b-5ff0cd37b507 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.796152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:14.837769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b4279-566b-409c-b5e4-f5ff489b4711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.838435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b4279-566b-409c-b5e4-f5ff489b4711 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.838713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b4279-566b-409c-b5e4-f5ff489b4711 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.838981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b4279-566b-409c-b5e4-f5ff489b4711 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.839776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b4279-566b-409c-b5e4-f5ff489b4711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.841297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b4279-566b-409c-b5e4-f5ff489b4711 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.852226 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1368/2716] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 20:01:14.856441 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab44cb-953a-4137-8f11-d3ace1c5fe6a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:14.861882 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:14.868386 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab44cb-953a-4137-8f11-d3ace1c5fe6a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:14.873426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffe6cb-4557-4a1d-8241-f57e249352d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:14.874337 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffe6cb-4557-4a1d-8241-f57e249352d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.874616 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffe6cb-4557-4a1d-8241-f57e249352d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.874883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffe6cb-4557-4a1d-8241-f57e249352d8 admin admin] PATH_INFO: `/v3/projects/8bc2bd402ee446b19bb05c41d61bc2af/users/6ff1126dfc014e91aa3535769ce44c3a/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.876823 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffe6cb-4557-4a1d-8241-f57e249352d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc2bd402ee446b19bb05c41d61bc2af, user_id=6ff1126dfc014e91aa3535769ce44c3a, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:14.883072 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffe6cb-4557-4a1d-8241-f57e249352d8 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:14.893359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1349/2717] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:01:14 2022] PUT /identity/v3/projects/8bc2bd402ee446b19bb05c41d61bc2af/users/6ff1126dfc014e91aa3535769ce44c3a/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:14.904576 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45afa-9c6f-498b-a116-2f9d38037f42 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:14.904803 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45afa-9c6f-498b-a116-2f9d38037f42 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:14.905083 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d45afa-9c6f-498b-a116-2f9d38037f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:14.932661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-e7d45afa-9c6f-498b-a116-2f9d38037f42 None None] MFA Rules not processed for user `6ff1126dfc014e91aa3535769ce44c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:14.933911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e7d45afa-9c6f-498b-a116-2f9d38037f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:15.001031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1369/2718] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:15.011777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffe6cb-4557-4a1d-8241-f57e249352d8 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:15.013501 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4bffe6cb-4557-4a1d-8241-f57e249352d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:15.071301 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffe6cb-4557-4a1d-8241-f57e249352d8 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:15.074915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbf3f5-feb1-4f53-9cfb-e881b27a5729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:15.075541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbf3f5-feb1-4f53-9cfb-e881b27a5729 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:15.075821 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbf3f5-feb1-4f53-9cfb-e881b27a5729 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:15.076094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbf3f5-feb1-4f53-9cfb-e881b27a5729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:15.139283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbf3f5-feb1-4f53-9cfb-e881b27a5729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:15.140713 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbf3f5-feb1-4f53-9cfb-e881b27a5729 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:15.152742 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1350/2719] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:01:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:01:19.310162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d45afa-9c6f-498b-a116-2f9d38037f42 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:19.311556 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-e7d45afa-9c6f-498b-a116-2f9d38037f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:19.323550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d45afa-9c6f-498b-a116-2f9d38037f42 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:19.327100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5d2ba-a356-40fc-94b0-e717c0d4b5f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:19.327737 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5d2ba-a356-40fc-94b0-e717c0d4b5f3 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:19.328019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5d2ba-a356-40fc-94b0-e717c0d4b5f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:19.328292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5d2ba-a356-40fc-94b0-e717c0d4b5f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:19.337827 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5d2ba-a356-40fc-94b0-e717c0d4b5f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:19.339249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5d2ba-a356-40fc-94b0-e717c0d4b5f3 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:19.349526 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1370/2720] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:01:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:01:24.194327 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbf3f5-feb1-4f53-9cfb-e881b27a5729 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:24.257556 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbf3f5-feb1-4f53-9cfb-e881b27a5729 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:24.262111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:24.262970 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:24.263253 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:24.263531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] PATH_INFO: `/v3/users/dffbeae0c97640299fa6366c45658cc1` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:24.264716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dffbeae0c97640299fa6366c45658cc1)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:24.266690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:24.313637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:24.315209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dffbeae0c97640299fa6366c45658cc1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:24.329375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dffbeae0c97640299fa6366c45658cc1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:24.337968 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dffbeae0c97640299fa6366c45658cc1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:24.344694 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dffbeae0c97640299fa6366c45658cc1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:24.345308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dffbeae0c97640299fa6366c45658cc1'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:24.352131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1351/2721] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:01:24 2022] DELETE /identity/v3/users/dffbeae0c97640299fa6366c45658cc1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:24.356729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b8b77-5eeb-4b8c-9bf9-8de905e3fa65 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:24.357243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b8b77-5eeb-4b8c-9bf9-8de905e3fa65 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:24.357351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b8b77-5eeb-4b8c-9bf9-8de905e3fa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:24.375200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5e6b8b77-5eeb-4b8c-9bf9-8de905e3fa65 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:24.376382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5e6b8b77-5eeb-4b8c-9bf9-8de905e3fa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:24.434627 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1371/2722] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:24.444207 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:24.445699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:24.502805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de6aa4-f9d8-44f5-80c8-95d2d74b198a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:24.506813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d95a6d-5108-467d-96ca-6aa7a4004681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:24.507457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d95a6d-5108-467d-96ca-6aa7a4004681 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:24.507738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d95a6d-5108-467d-96ca-6aa7a4004681 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:24.508010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d95a6d-5108-467d-96ca-6aa7a4004681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:24.517277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d95a6d-5108-467d-96ca-6aa7a4004681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:24.519060 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d95a6d-5108-467d-96ca-6aa7a4004681 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:24.529376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1352/2723] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:01:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:01:24.794286 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b8b77-5eeb-4b8c-9bf9-8de905e3fa65 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:24.795719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5e6b8b77-5eeb-4b8c-9bf9-8de905e3fa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:24.808128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b8b77-5eeb-4b8c-9bf9-8de905e3fa65 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:24.813464 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:24.817744 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:24.818069 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:24.818353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] PATH_INFO: `/v3/projects/2c4f04845f1b4987b29cafc3971c75d6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:24.821201 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c4f04845f1b4987b29cafc3971c75d6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:24.823323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:24.860245 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:24.877404 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c4f04845f1b4987b29cafc3971c75d6', 'request_id': 'req-cea481eb-91dc-4a78-8068-b069b21df988'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:24.885470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c4f04845f1b4987b29cafc3971c75d6', 'request_id': 'req-cea481eb-91dc-4a78-8068-b069b21df988'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:24.891731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c4f04845f1b4987b29cafc3971c75d6', 'request_id': 'req-cea481eb-91dc-4a78-8068-b069b21df988'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:24.902630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] The token cache is being invalidate because project 2c4f04845f1b4987b29cafc3971c75d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:01:24.902915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:24.903551 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:24.905080 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1372/2724] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:01:24 2022] DELETE /identity/v3/projects/2c4f04845f1b4987b29cafc3971c75d6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:24.909112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d95a6d-5108-467d-96ca-6aa7a4004681 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:24.919147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d95a6d-5108-467d-96ca-6aa7a4004681 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:24.976069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:24.976874 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:24.977153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:24.977420 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] PATH_INFO: `/v3/users/6ff1126dfc014e91aa3535769ce44c3a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:24.978699 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff1126dfc014e91aa3535769ce44c3a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:24.980618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:25.025935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff1126dfc014e91aa3535769ce44c3a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:25.038122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff1126dfc014e91aa3535769ce44c3a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:25.040590 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:25.045898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff1126dfc014e91aa3535769ce44c3a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:25.054136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff1126dfc014e91aa3535769ce44c3a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:25.054776 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-9fa3445d-8324-4dcc-a7ff-885e10554234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff1126dfc014e91aa3535769ce44c3a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:25.060680 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1353/2725] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:01:24 2022] DELETE /identity/v3/users/6ff1126dfc014e91aa3535769ce44c3a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:25.329050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:25.330524 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:25.345419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea481eb-91dc-4a78-8068-b069b21df988 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:25.403216 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:25.404085 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:25.404366 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:25.404640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] PATH_INFO: `/v3/projects/8bc2bd402ee446b19bb05c41d61bc2af` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:25.405841 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bc2bd402ee446b19bb05c41d61bc2af)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:25.407734 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:25.452947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bc2bd402ee446b19bb05c41d61bc2af', 'request_id': 'req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:25.459942 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bc2bd402ee446b19bb05c41d61bc2af', 'request_id': 'req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:25.465901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bc2bd402ee446b19bb05c41d61bc2af', 'request_id': 'req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:25.476263 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] The token cache is being invalidate because project 8bc2bd402ee446b19bb05c41d61bc2af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:01:25.476545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:25.477170 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7c677466-ddc8-4aa8-a5b7-3c2b258451fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:25.478713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1373/2726] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:01:25 2022] DELETE /identity/v3/projects/8bc2bd402ee446b19bb05c41d61bc2af => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:25.484767 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7f383-cb59-47d8-a028-050c8aec7523 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:25.485224 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7f383-cb59-47d8-a028-050c8aec7523 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:25.485335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7f383-cb59-47d8-a028-050c8aec7523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:25.504872 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-5dd7f383-cb59-47d8-a028-050c8aec7523 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:25.506124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-5dd7f383-cb59-47d8-a028-050c8aec7523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:25.564993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1354/2727] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:25.573984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57899e-1ceb-473b-bde8-e2b2bc1f5320 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:25.574269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57899e-1ceb-473b-bde8-e2b2bc1f5320 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:25.574545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57899e-1ceb-473b-bde8-e2b2bc1f5320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:25.592801 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-4e57899e-1ceb-473b-bde8-e2b2bc1f5320 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:25.594037 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4e57899e-1ceb-473b-bde8-e2b2bc1f5320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:25.602795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1374/2728] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:25.607069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7f383-cb59-47d8-a028-050c8aec7523 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:25.617472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7f383-cb59-47d8-a028-050c8aec7523 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:25.621997 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a519-42dd-4d00-b78d-4506b9d64da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:25.622641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a519-42dd-4d00-b78d-4506b9d64da4 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:25.622918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a519-42dd-4d00-b78d-4506b9d64da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:25.623187 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a519-42dd-4d00-b78d-4506b9d64da4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:25.623901 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a519-42dd-4d00-b78d-4506b9d64da4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:01:25.624199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a519-42dd-4d00-b78d-4506b9d64da4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:25.625804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a519-42dd-4d00-b78d-4506b9d64da4 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:25.637246 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1355/2729] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 20:01:25.641619 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57899e-1ceb-473b-bde8-e2b2bc1f5320 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:25.651303 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57899e-1ceb-473b-bde8-e2b2bc1f5320 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:25.657954 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb905a-72e7-4655-a5f7-b39d73d78b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:25.658613 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb905a-72e7-4655-a5f7-b39d73d78b7f admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:25.658886 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb905a-72e7-4655-a5f7-b39d73d78b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:25.659154 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb905a-72e7-4655-a5f7-b39d73d78b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:25.659845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb905a-72e7-4655-a5f7-b39d73d78b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:25.662278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb905a-72e7-4655-a5f7-b39d73d78b7f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:25.688243 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1375/2730] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:01:25 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:01:25.692825 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a519-42dd-4d00-b78d-4506b9d64da4 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:25.702745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a519-42dd-4d00-b78d-4506b9d64da4 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:25.759442 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345301e-0c70-467a-9a94-1a7f7b2dbf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:25.760028 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345301e-0c70-467a-9a94-1a7f7b2dbf5e admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:25.760306 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345301e-0c70-467a-9a94-1a7f7b2dbf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:25.760630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345301e-0c70-467a-9a94-1a7f7b2dbf5e admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:25.761308 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345301e-0c70-467a-9a94-1a7f7b2dbf5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:25.763249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345301e-0c70-467a-9a94-1a7f7b2dbf5e admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:25.782925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1356/2731] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:01:25 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:01:25.787544 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb905a-72e7-4655-a5f7-b39d73d78b7f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:25.797515 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb905a-72e7-4655-a5f7-b39d73d78b7f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:25.802769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f99d3-347d-40f2-9324-ee2f1b2e43af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:25.803413 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f99d3-347d-40f2-9324-ee2f1b2e43af admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:25.803694 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f99d3-347d-40f2-9324-ee2f1b2e43af admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:25.803966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f99d3-347d-40f2-9324-ee2f1b2e43af admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:25.804704 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f99d3-347d-40f2-9324-ee2f1b2e43af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:25.806341 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f99d3-347d-40f2-9324-ee2f1b2e43af admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:25.816457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1376/2732] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:01:25.820196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345301e-0c70-467a-9a94-1a7f7b2dbf5e admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:25.830239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345301e-0c70-467a-9a94-1a7f7b2dbf5e admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:25.835099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062bda9-998f-4cf5-860d-30de2f194960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:25.835755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062bda9-998f-4cf5-860d-30de2f194960 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:25.836038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062bda9-998f-4cf5-860d-30de2f194960 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:25.836313 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062bda9-998f-4cf5-860d-30de2f194960 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:25.837034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062bda9-998f-4cf5-860d-30de2f194960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:25.838747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062bda9-998f-4cf5-860d-30de2f194960 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:25.849255 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1357/2733] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:01:25.853557 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f99d3-347d-40f2-9324-ee2f1b2e43af admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:25.864000 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f99d3-347d-40f2-9324-ee2f1b2e43af admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:25.868605 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b944ff9-ced8-41b1-98dd-8c01c9db35d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:25.869488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944ff9-ced8-41b1-98dd-8c01c9db35d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:25.869952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944ff9-ced8-41b1-98dd-8c01c9db35d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:25.870229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b944ff9-ced8-41b1-98dd-8c01c9db35d7 admin admin] PATH_INFO: `/v3/projects/7a2cf919632e466fa55d34aa94e3e18e/users/54a365cd7d384d22b8bbe11ae395f156/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:25.872462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:25.887663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b944ff9-ced8-41b1-98dd-8c01c9db35d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a2cf919632e466fa55d34aa94e3e18e, user_id=54a365cd7d384d22b8bbe11ae395f156, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:25.890383 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b944ff9-ced8-41b1-98dd-8c01c9db35d7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:25.902925 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1377/2734] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:01:25 2022] PUT /identity/v3/projects/7a2cf919632e466fa55d34aa94e3e18e/users/54a365cd7d384d22b8bbe11ae395f156/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:25.914615 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d643e-b7dd-4d96-9776-2aa2269a54ea None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:25.914914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d643e-b7dd-4d96-9776-2aa2269a54ea None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:25.915198 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d643e-b7dd-4d96-9776-2aa2269a54ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:25.942375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-f87d643e-b7dd-4d96-9776-2aa2269a54ea None None] MFA Rules not processed for user `54a365cd7d384d22b8bbe11ae395f156`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:25.943629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f87d643e-b7dd-4d96-9776-2aa2269a54ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:25.984237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:26.010216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1358/2735] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:26.014582 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b944ff9-ced8-41b1-98dd-8c01c9db35d7 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:26.079571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b944ff9-ced8-41b1-98dd-8c01c9db35d7 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:26.086468 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d947ae4-b8f1-4b2a-8ac1-5f9a8a88d7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:26.087235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d947ae4-b8f1-4b2a-8ac1-5f9a8a88d7ff admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:26.087606 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d947ae4-b8f1-4b2a-8ac1-5f9a8a88d7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:26.087947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d947ae4-b8f1-4b2a-8ac1-5f9a8a88d7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:26.088715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d947ae4-b8f1-4b2a-8ac1-5f9a8a88d7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:26.090778 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d947ae4-b8f1-4b2a-8ac1-5f9a8a88d7ff admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:26.117861 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1378/2736] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:01:26 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:01:26.122086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d643e-b7dd-4d96-9776-2aa2269a54ea None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:26.133275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d643e-b7dd-4d96-9776-2aa2269a54ea None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:26.190988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6f40c-09d0-47de-8579-787242976896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:26.192112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6f40c-09d0-47de-8579-787242976896 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:26.192498 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6f40c-09d0-47de-8579-787242976896 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:26.192785 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6f40c-09d0-47de-8579-787242976896 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:26.193493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6f40c-09d0-47de-8579-787242976896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:26.195468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6f40c-09d0-47de-8579-787242976896 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:26.215963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1359/2737] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:01:26 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:01:26.219865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d947ae4-b8f1-4b2a-8ac1-5f9a8a88d7ff admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:26.232050 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d947ae4-b8f1-4b2a-8ac1-5f9a8a88d7ff admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:26.236648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded3158-5170-42b4-9111-fe88919ae98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:26.237328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded3158-5170-42b4-9111-fe88919ae98f admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:26.237630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded3158-5170-42b4-9111-fe88919ae98f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:26.237913 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded3158-5170-42b4-9111-fe88919ae98f admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:26.238660 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded3158-5170-42b4-9111-fe88919ae98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:26.240264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded3158-5170-42b4-9111-fe88919ae98f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:26.251514 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1379/2738] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:01:26.255903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6f40c-09d0-47de-8579-787242976896 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:26.265962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6f40c-09d0-47de-8579-787242976896 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:26.271250 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518349e-bff9-4c43-a270-bdc32b393ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:26.272118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518349e-bff9-4c43-a270-bdc32b393ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:26.272400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518349e-bff9-4c43-a270-bdc32b393ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:26.272675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518349e-bff9-4c43-a270-bdc32b393ce5 admin admin] PATH_INFO: `/v3/projects/9d0befe2b063438f861c1a711db988ee/users/73605691408c4f3bb7d189d8bfc8e330/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:26.287675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518349e-bff9-4c43-a270-bdc32b393ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0befe2b063438f861c1a711db988ee, user_id=73605691408c4f3bb7d189d8bfc8e330, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:26.290342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518349e-bff9-4c43-a270-bdc32b393ce5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:26.303729 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1360/2739] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:01:26 2022] PUT /identity/v3/projects/9d0befe2b063438f861c1a711db988ee/users/73605691408c4f3bb7d189d8bfc8e330/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:26.307816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded3158-5170-42b4-9111-fe88919ae98f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:26.317759 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded3158-5170-42b4-9111-fe88919ae98f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:26.373323 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19571c-2188-4601-a9ed-2f050fed3f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:26.374018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19571c-2188-4601-a9ed-2f050fed3f6d admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:26.374910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19571c-2188-4601-a9ed-2f050fed3f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:26.375196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19571c-2188-4601-a9ed-2f050fed3f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:26.375917 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19571c-2188-4601-a9ed-2f050fed3f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:26.377535 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19571c-2188-4601-a9ed-2f050fed3f6d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:26.387790 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1380/2740] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:01:26.392195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518349e-bff9-4c43-a270-bdc32b393ce5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:26.403402 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518349e-bff9-4c43-a270-bdc32b393ce5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:26.408011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ec0f2-4732-4cca-b40d-f24be57bdc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:26.408946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec0f2-4732-4cca-b40d-f24be57bdc0d admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:26.409229 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec0f2-4732-4cca-b40d-f24be57bdc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:26.409505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec0f2-4732-4cca-b40d-f24be57bdc0d admin admin] PATH_INFO: `/v3/projects/9d0befe2b063438f861c1a711db988ee/users/73605691408c4f3bb7d189d8bfc8e330/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:26.411492 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ec0f2-4732-4cca-b40d-f24be57bdc0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0befe2b063438f861c1a711db988ee, user_id=73605691408c4f3bb7d189d8bfc8e330, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:26.414177 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ec0f2-4732-4cca-b40d-f24be57bdc0d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:26.424277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1361/2741] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:01:26 2022] PUT /identity/v3/projects/9d0befe2b063438f861c1a711db988ee/users/73605691408c4f3bb7d189d8bfc8e330/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:26.435816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15efb59-a044-4a1b-87dc-be2392dba41c None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:26.436333 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15efb59-a044-4a1b-87dc-be2392dba41c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:26.436458 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15efb59-a044-4a1b-87dc-be2392dba41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:26.462794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f15efb59-a044-4a1b-87dc-be2392dba41c None None] MFA Rules not processed for user `73605691408c4f3bb7d189d8bfc8e330`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:26.464021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f15efb59-a044-4a1b-87dc-be2392dba41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:26.505659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:26.533646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1381/2742] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:26.544585 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ec0f2-4732-4cca-b40d-f24be57bdc0d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:26.546331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-376ec0f2-4732-4cca-b40d-f24be57bdc0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:26.604589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ec0f2-4732-4cca-b40d-f24be57bdc0d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:26.608181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d771f8-1c9b-4a7d-8048-aa401960ffd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:26.608806 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d771f8-1c9b-4a7d-8048-aa401960ffd5 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:26.609084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d771f8-1c9b-4a7d-8048-aa401960ffd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:26.609353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d771f8-1c9b-4a7d-8048-aa401960ffd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:26.654176 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:26.668242 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d771f8-1c9b-4a7d-8048-aa401960ffd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:26.669802 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d771f8-1c9b-4a7d-8048-aa401960ffd5 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:26.679500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1362/2743] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:01:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:01:29.221104 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15efb59-a044-4a1b-87dc-be2392dba41c None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:29.281674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15efb59-a044-4a1b-87dc-be2392dba41c None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:29.286235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:29.287153 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:29.287438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:29.287746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] PATH_INFO: `/v3/users/54a365cd7d384d22b8bbe11ae395f156` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:29.288984 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a365cd7d384d22b8bbe11ae395f156)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:29.290939 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:29.340506 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a365cd7d384d22b8bbe11ae395f156'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:29.349123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a365cd7d384d22b8bbe11ae395f156'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:29.349772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a365cd7d384d22b8bbe11ae395f156'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:29.362305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a365cd7d384d22b8bbe11ae395f156'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:29.362940 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a365cd7d384d22b8bbe11ae395f156'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:29.375380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1382/2744] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:01:29 2022] DELETE /identity/v3/users/54a365cd7d384d22b8bbe11ae395f156 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:29.380180 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaa2c3-7177-4509-9e25-635fb292d64f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:29.380671 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaa2c3-7177-4509-9e25-635fb292d64f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:29.380787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddaa2c3-7177-4509-9e25-635fb292d64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:29.398191 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-1ddaa2c3-7177-4509-9e25-635fb292d64f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:29.399387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1ddaa2c3-7177-4509-9e25-635fb292d64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:29.460166 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1363/2745] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:29.469695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:29.471688 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:29.529169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999134d-6d54-4b4a-866e-d160c6822b3e admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:29.532782 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b52cf-7cc7-4e69-95cc-7f6b4af459e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:29.533415 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b52cf-7cc7-4e69-95cc-7f6b4af459e9 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:29.533730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b52cf-7cc7-4e69-95cc-7f6b4af459e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:29.534003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b52cf-7cc7-4e69-95cc-7f6b4af459e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:29.543359 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b52cf-7cc7-4e69-95cc-7f6b4af459e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:29.544786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b52cf-7cc7-4e69-95cc-7f6b4af459e9 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:29.549826 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:29.555614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1383/2746] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:01:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:01:29.823887 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddaa2c3-7177-4509-9e25-635fb292d64f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:29.825369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1ddaa2c3-7177-4509-9e25-635fb292d64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:29.838695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddaa2c3-7177-4509-9e25-635fb292d64f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:29.843293 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:29.844155 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:29.844433 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:29.844733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] PATH_INFO: `/v3/projects/7a2cf919632e466fa55d34aa94e3e18e` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:29.845947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a2cf919632e466fa55d34aa94e3e18e)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:29.848645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:29.895112 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a2cf919632e466fa55d34aa94e3e18e', 'request_id': 'req-69e042df-a8a5-4de3-8e15-70f17250db3a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:29.902496 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a2cf919632e466fa55d34aa94e3e18e', 'request_id': 'req-69e042df-a8a5-4de3-8e15-70f17250db3a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:29.908335 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a2cf919632e466fa55d34aa94e3e18e', 'request_id': 'req-69e042df-a8a5-4de3-8e15-70f17250db3a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:29.920685 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] The token cache is being invalidate because project 7a2cf919632e466fa55d34aa94e3e18e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:01:29.920975 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:29.921697 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:29.923251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1364/2747] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:01:29 2022] DELETE /identity/v3/projects/7a2cf919632e466fa55d34aa94e3e18e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:29.927368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b52cf-7cc7-4e69-95cc-7f6b4af459e9 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:29.936910 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b52cf-7cc7-4e69-95cc-7f6b4af459e9 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:29.995159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:29.995958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:29.996235 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:29.996505 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] PATH_INFO: `/v3/users/73605691408c4f3bb7d189d8bfc8e330` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:29.998152 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73605691408c4f3bb7d189d8bfc8e330)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:30.000120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:30.048127 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73605691408c4f3bb7d189d8bfc8e330'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:30.055681 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73605691408c4f3bb7d189d8bfc8e330'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:30.056289 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73605691408c4f3bb7d189d8bfc8e330'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:30.068326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73605691408c4f3bb7d189d8bfc8e330'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:30.068952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-771e40e7-17b9-4244-8259-33670f4ac7d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73605691408c4f3bb7d189d8bfc8e330'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:30.080808 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1384/2748] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:01:29 2022] DELETE /identity/v3/users/73605691408c4f3bb7d189d8bfc8e330 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:30.369669 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:30.371219 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:30.385463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e042df-a8a5-4de3-8e15-70f17250db3a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:30.443637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:30.444497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:30.444772 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:30.445042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] PATH_INFO: `/v3/projects/9d0befe2b063438f861c1a711db988ee` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:30.446223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0befe2b063438f861c1a711db988ee)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:30.448109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:30.460200 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:30.496360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0befe2b063438f861c1a711db988ee', 'request_id': 'req-10731eaa-6deb-4134-b6f0-b2660f519b75'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:30.504570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0befe2b063438f861c1a711db988ee', 'request_id': 'req-10731eaa-6deb-4134-b6f0-b2660f519b75'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:30.510755 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0befe2b063438f861c1a711db988ee', 'request_id': 'req-10731eaa-6deb-4134-b6f0-b2660f519b75'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:30.523638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] The token cache is being invalidate because project 9d0befe2b063438f861c1a711db988ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:01:30.523929 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:30.524579 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-10731eaa-6deb-4134-b6f0-b2660f519b75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:30.526154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1365/2749] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:01:30 2022] DELETE /identity/v3/projects/9d0befe2b063438f861c1a711db988ee => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:30.532218 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25173-d2c0-4bc8-bece-26332e13edf5 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:30.532671 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25173-d2c0-4bc8-bece-26332e13edf5 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:30.532769 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25173-d2c0-4bc8-bece-26332e13edf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:30.552297 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-a5e25173-d2c0-4bc8-bece-26332e13edf5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:30.553502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a5e25173-d2c0-4bc8-bece-26332e13edf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:30.614651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1385/2750] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:30.622725 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044eddf-fe35-4548-a85c-6b8146bbd783 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:30.623084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044eddf-fe35-4548-a85c-6b8146bbd783 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:30.623455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044eddf-fe35-4548-a85c-6b8146bbd783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:30.641474 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-1044eddf-fe35-4548-a85c-6b8146bbd783 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:30.642961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1044eddf-fe35-4548-a85c-6b8146bbd783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:30.651424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1366/2751] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:30.655665 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e25173-d2c0-4bc8-bece-26332e13edf5 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:30.666439 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e25173-d2c0-4bc8-bece-26332e13edf5 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:30.671463 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c3b1d-4e45-4e19-bc6a-f3704690c2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:30.672125 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c3b1d-4e45-4e19-bc6a-f3704690c2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:30.672408 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c3b1d-4e45-4e19-bc6a-f3704690c2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:30.672680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c3b1d-4e45-4e19-bc6a-f3704690c2e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:30.673450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c3b1d-4e45-4e19-bc6a-f3704690c2e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:01:30.673795 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c3b1d-4e45-4e19-bc6a-f3704690c2e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:30.675396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c3b1d-4e45-4e19-bc6a-f3704690c2e0 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:30.687291 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1386/2752] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 20:01:30.691662 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044eddf-fe35-4548-a85c-6b8146bbd783 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:30.700881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044eddf-fe35-4548-a85c-6b8146bbd783 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:30.705726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe4153-48ab-41fa-b4ee-8cc0839e2db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:30.706377 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe4153-48ab-41fa-b4ee-8cc0839e2db0 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:30.706661 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe4153-48ab-41fa-b4ee-8cc0839e2db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:30.707651 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe4153-48ab-41fa-b4ee-8cc0839e2db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:30.708204 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe4153-48ab-41fa-b4ee-8cc0839e2db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:30.710126 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe4153-48ab-41fa-b4ee-8cc0839e2db0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:30.735220 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1367/2753] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:01:30 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:01:30.739712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c3b1d-4e45-4e19-bc6a-f3704690c2e0 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:30.749998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c3b1d-4e45-4e19-bc6a-f3704690c2e0 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:30.753520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:30.809729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703d0ea-be13-4670-b18c-569970bd0e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:30.810397 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703d0ea-be13-4670-b18c-569970bd0e6f admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:30.810678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703d0ea-be13-4670-b18c-569970bd0e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:30.810952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703d0ea-be13-4670-b18c-569970bd0e6f admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:30.811634 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703d0ea-be13-4670-b18c-569970bd0e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:30.813611 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703d0ea-be13-4670-b18c-569970bd0e6f admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:30.833302 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1387/2754] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:01:30 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:01:30.838361 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe4153-48ab-41fa-b4ee-8cc0839e2db0 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:30.850934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe4153-48ab-41fa-b4ee-8cc0839e2db0 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:30.855554 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5e4ca-baf9-4270-aa6b-b978f88237a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:30.856203 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5e4ca-baf9-4270-aa6b-b978f88237a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:30.856488 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5e4ca-baf9-4270-aa6b-b978f88237a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:30.856758 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5e4ca-baf9-4270-aa6b-b978f88237a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:30.857503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5e4ca-baf9-4270-aa6b-b978f88237a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:30.859163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5e4ca-baf9-4270-aa6b-b978f88237a0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:30.869855 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1368/2755] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 20:01:30.873809 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703d0ea-be13-4670-b18c-569970bd0e6f admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:30.884232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703d0ea-be13-4670-b18c-569970bd0e6f admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:30.888966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506dd7e-d7a8-425a-afb2-d70c8a0ae58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:30.889755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506dd7e-d7a8-425a-afb2-d70c8a0ae58a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:30.890058 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506dd7e-d7a8-425a-afb2-d70c8a0ae58a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:30.890328 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506dd7e-d7a8-425a-afb2-d70c8a0ae58a admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:30.891044 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506dd7e-d7a8-425a-afb2-d70c8a0ae58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:30.892877 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506dd7e-d7a8-425a-afb2-d70c8a0ae58a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:30.903703 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1388/2756] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:01:30.907761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5e4ca-baf9-4270-aa6b-b978f88237a0 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:30.917752 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5e4ca-baf9-4270-aa6b-b978f88237a0 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:30.923214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78170e3-56ab-4ea9-b8ce-546147c85fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:30.924024 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78170e3-56ab-4ea9-b8ce-546147c85fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:30.924303 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78170e3-56ab-4ea9-b8ce-546147c85fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:30.924581 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78170e3-56ab-4ea9-b8ce-546147c85fa7 admin admin] PATH_INFO: `/v3/projects/aaa32fffc74548759edc8a9238d00859/users/f6925068d1584d2a9d9639c1ad787bfd/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:30.939068 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78170e3-56ab-4ea9-b8ce-546147c85fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa32fffc74548759edc8a9238d00859, user_id=f6925068d1584d2a9d9639c1ad787bfd, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:30.942446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78170e3-56ab-4ea9-b8ce-546147c85fa7 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:30.955023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1369/2757] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:01:30 2022] PUT /identity/v3/projects/aaa32fffc74548759edc8a9238d00859/users/f6925068d1584d2a9d9639c1ad787bfd/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:30.965917 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a812d-c31f-484d-8556-2530626b1419 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:30.966270 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a812d-c31f-484d-8556-2530626b1419 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:30.966620 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a812d-c31f-484d-8556-2530626b1419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:30.992843 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-774a812d-c31f-484d-8556-2530626b1419 None None] MFA Rules not processed for user `f6925068d1584d2a9d9639c1ad787bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:30.994097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-774a812d-c31f-484d-8556-2530626b1419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:31.057630 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1389/2758] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:31.069258 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78170e3-56ab-4ea9-b8ce-546147c85fa7 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:31.070973 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a78170e3-56ab-4ea9-b8ce-546147c85fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:31.126589 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78170e3-56ab-4ea9-b8ce-546147c85fa7 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:31.130928 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047df600-6014-470c-b0d6-9ea9ab040741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:31.131554 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047df600-6014-470c-b0d6-9ea9ab040741 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:31.131858 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047df600-6014-470c-b0d6-9ea9ab040741 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:31.132128 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047df600-6014-470c-b0d6-9ea9ab040741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:31.140315 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047df600-6014-470c-b0d6-9ea9ab040741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:31.141837 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047df600-6014-470c-b0d6-9ea9ab040741 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:31.151597 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1370/2759] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:01:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:01:33.682431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a812d-c31f-484d-8556-2530626b1419 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:33.747669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a812d-c31f-484d-8556-2530626b1419 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:33.753006 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:33.754086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:33.754086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:33.760844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] PATH_INFO: `/v3/users/f6925068d1584d2a9d9639c1ad787bfd` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:33.760844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6925068d1584d2a9d9639c1ad787bfd)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:33.760844 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:33.777791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:33.808005 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6925068d1584d2a9d9639c1ad787bfd'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:33.814920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6925068d1584d2a9d9639c1ad787bfd'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:33.815525 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6925068d1584d2a9d9639c1ad787bfd'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:33.827188 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6925068d1584d2a9d9639c1ad787bfd'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:33.827814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6925068d1584d2a9d9639c1ad787bfd'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:33.840853 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1390/2760] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:01:33 2022] DELETE /identity/v3/users/f6925068d1584d2a9d9639c1ad787bfd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:33.845147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3330c-6472-4e58-8815-ed8bec03bc50 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:33.845675 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3330c-6472-4e58-8815-ed8bec03bc50 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:33.845782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3330c-6472-4e58-8815-ed8bec03bc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:33.870542 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-4af3330c-6472-4e58-8815-ed8bec03bc50 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:33.871753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4af3330c-6472-4e58-8815-ed8bec03bc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:33.906602 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:33.934008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1371/2761] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:33.944195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:33.945612 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:34.003252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a0fd0-57a9-4776-85b8-0856a48bec51 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:34.006897 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e6fbf-2b6d-45d2-a93f-38c94271bf0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:34.007541 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6fbf-2b6d-45d2-a93f-38c94271bf0e service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.007823 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6fbf-2b6d-45d2-a93f-38c94271bf0e service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.008099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6fbf-2b6d-45d2-a93f-38c94271bf0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.017520 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e6fbf-2b6d-45d2-a93f-38c94271bf0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:34.018991 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e6fbf-2b6d-45d2-a93f-38c94271bf0e service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:34.028800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1391/2762] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:01:34.291419 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3330c-6472-4e58-8815-ed8bec03bc50 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:34.292848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-4af3330c-6472-4e58-8815-ed8bec03bc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:34.310781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3330c-6472-4e58-8815-ed8bec03bc50 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:34.315421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:34.316285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.316667 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.317247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] PATH_INFO: `/v3/projects/aaa32fffc74548759edc8a9238d00859` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.318359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa32fffc74548759edc8a9238d00859)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:34.320254 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:34.368457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa32fffc74548759edc8a9238d00859', 'request_id': 'req-85cda992-9224-43f2-94b3-d99c79f1ffa9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:34.375851 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa32fffc74548759edc8a9238d00859', 'request_id': 'req-85cda992-9224-43f2-94b3-d99c79f1ffa9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:34.381710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa32fffc74548759edc8a9238d00859', 'request_id': 'req-85cda992-9224-43f2-94b3-d99c79f1ffa9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:34.393197 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] The token cache is being invalidate because project aaa32fffc74548759edc8a9238d00859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:01:34.393486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:34.394826 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-85cda992-9224-43f2-94b3-d99c79f1ffa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:01:34.396431 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1372/2763] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:01:34 2022] DELETE /identity/v3/projects/aaa32fffc74548759edc8a9238d00859 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:01:34.402004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4df1a-a9cf-405c-81b9-e271c96f98aa None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.402381 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4df1a-a9cf-405c-81b9-e271c96f98aa None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.402689 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4df1a-a9cf-405c-81b9-e271c96f98aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.420932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9bf4df1a-a9cf-405c-81b9-e271c96f98aa None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:34.422162 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9bf4df1a-a9cf-405c-81b9-e271c96f98aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:34.480280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1392/2764] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 20:01:34.542376 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e4f2c-accc-4cb6-bda5-8102dbe23a54 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.542696 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e4f2c-accc-4cb6-bda5-8102dbe23a54 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.542978 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e4f2c-accc-4cb6-bda5-8102dbe23a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.561803 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-202e4f2c-accc-4cb6-bda5-8102dbe23a54 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:34.563037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-202e4f2c-accc-4cb6-bda5-8102dbe23a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:34.571029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1373/2765] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:34.575363 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4df1a-a9cf-405c-81b9-e271c96f98aa None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:34.585422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4df1a-a9cf-405c-81b9-e271c96f98aa None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:34.590029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d553bb-d7c0-4d0d-8427-8fca4e3e5520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:34.590678 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d553bb-d7c0-4d0d-8427-8fca4e3e5520 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.590959 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d553bb-d7c0-4d0d-8427-8fca4e3e5520 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.591234 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d553bb-d7c0-4d0d-8427-8fca4e3e5520 admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.591946 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d553bb-d7c0-4d0d-8427-8fca4e3e5520 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:01:34.592255 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d553bb-d7c0-4d0d-8427-8fca4e3e5520 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:34.593874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d553bb-d7c0-4d0d-8427-8fca4e3e5520 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:34.606045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1393/2766] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 20:01:34.610321 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e4f2c-accc-4cb6-bda5-8102dbe23a54 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:34.619505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e4f2c-accc-4cb6-bda5-8102dbe23a54 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:34.624090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226d790-660d-481d-aec3-3b6425c88f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:34.624728 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226d790-660d-481d-aec3-3b6425c88f1a admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.625011 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226d790-660d-481d-aec3-3b6425c88f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.625287 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226d790-660d-481d-aec3-3b6425c88f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.626059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226d790-660d-481d-aec3-3b6425c88f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:34.627889 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226d790-660d-481d-aec3-3b6425c88f1a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:34.654655 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1374/2767] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:01:34 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:01:34.658728 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d553bb-d7c0-4d0d-8427-8fca4e3e5520 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:34.669016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d553bb-d7c0-4d0d-8427-8fca4e3e5520 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:34.709722 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:34.747137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079b160-3b1b-4901-b8a2-adaf650117c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:34.747799 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b160-3b1b-4901-b8a2-adaf650117c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.748077 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b160-3b1b-4901-b8a2-adaf650117c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.748348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b160-3b1b-4901-b8a2-adaf650117c7 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.749019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079b160-3b1b-4901-b8a2-adaf650117c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:34.750935 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079b160-3b1b-4901-b8a2-adaf650117c7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:34.770850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1394/2768] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:01:34 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:01:34.775109 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226d790-660d-481d-aec3-3b6425c88f1a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:34.785545 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226d790-660d-481d-aec3-3b6425c88f1a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:34.790163 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329c7e1-de69-4463-9e8e-6ecda0226d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:34.790818 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329c7e1-de69-4463-9e8e-6ecda0226d4c admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.791098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329c7e1-de69-4463-9e8e-6ecda0226d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.791375 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329c7e1-de69-4463-9e8e-6ecda0226d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.792098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329c7e1-de69-4463-9e8e-6ecda0226d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:34.793715 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329c7e1-de69-4463-9e8e-6ecda0226d4c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:34.804338 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1375/2769] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:01:34.808179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079b160-3b1b-4901-b8a2-adaf650117c7 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:34.819266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079b160-3b1b-4901-b8a2-adaf650117c7 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:34.824078 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45053371-bfec-48cb-9aff-e1d6285dc03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:34.824768 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053371-bfec-48cb-9aff-e1d6285dc03c admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.825045 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053371-bfec-48cb-9aff-e1d6285dc03c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.825317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45053371-bfec-48cb-9aff-e1d6285dc03c admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.826073 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45053371-bfec-48cb-9aff-e1d6285dc03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:34.827658 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45053371-bfec-48cb-9aff-e1d6285dc03c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:34.838494 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1395/2770] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:01:34.843008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329c7e1-de69-4463-9e8e-6ecda0226d4c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:34.853199 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329c7e1-de69-4463-9e8e-6ecda0226d4c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:34.857931 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5da38-49e4-4778-a694-7a548aa787f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:34.858804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5da38-49e4-4778-a694-7a548aa787f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.859089 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5da38-49e4-4778-a694-7a548aa787f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.859366 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5da38-49e4-4778-a694-7a548aa787f5 admin admin] PATH_INFO: `/v3/projects/299f1631bc9542aea0ae0362f6e32a89/users/c7fe59652eb24e17bfc4572a8b713f7a/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.873594 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5da38-49e4-4778-a694-7a548aa787f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=299f1631bc9542aea0ae0362f6e32a89, user_id=c7fe59652eb24e17bfc4572a8b713f7a, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:34.876296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5da38-49e4-4778-a694-7a548aa787f5 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:34.889348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1376/2771] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:01:34 2022] PUT /identity/v3/projects/299f1631bc9542aea0ae0362f6e32a89/users/c7fe59652eb24e17bfc4572a8b713f7a/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 20:01:34.900453 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77f207-0b5f-4001-9149-43aec84031ef None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:34.900741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77f207-0b5f-4001-9149-43aec84031ef None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:34.901016 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77f207-0b5f-4001-9149-43aec84031ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:34.926438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7d77f207-0b5f-4001-9149-43aec84031ef None None] MFA Rules not processed for user `c7fe59652eb24e17bfc4572a8b713f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:01:34.927651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7d77f207-0b5f-4001-9149-43aec84031ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:34.995429 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1396/2772] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:01:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:01:35.007599 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5da38-49e4-4778-a694-7a548aa787f5 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:01:35.009275 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f7b5da38-49e4-4778-a694-7a548aa787f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:01:35.065472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5da38-49e4-4778-a694-7a548aa787f5 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:01:35.069784 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a35a3a-6861-42b3-a3d8-7360e0f853f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c022224b03a54ab489d95502c63a25dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:01:35.070438 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a35a3a-6861-42b3-a3d8-7360e0f853f8 service glance] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:01:35.070703 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a35a3a-6861-42b3-a3d8-7360e0f853f8 service glance] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:01:35.070982 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a35a3a-6861-42b3-a3d8-7360e0f853f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:01:35.079385 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a35a3a-6861-42b3-a3d8-7360e0f853f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:01:35.080813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a35a3a-6861-42b3-a3d8-7360e0f853f8 service glance] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:01:35.090437 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1377/2773] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:01:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:01:44.992590 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:01:45.086369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:02:01.269878 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ec7f2-8548-4f62-b503-eb1e82b7ef25 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:01.270325 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ec7f2-8548-4f62-b503-eb1e82b7ef25 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:01.270441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ec7f2-8548-4f62-b503-eb1e82b7ef25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:01.289550 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-c26ec7f2-8548-4f62-b503-eb1e82b7ef25 None None] MFA Rules not processed for user `8f2b9a6934d34f58a09a4cefb154371c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:02:01.290794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-c26ec7f2-8548-4f62-b503-eb1e82b7ef25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:01.345653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1397/2774] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:02:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:02:03.827236 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a35a3a-6861-42b3-a3d8-7360e0f853f8 service glance] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:03.898636 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a35a3a-6861-42b3-a3d8-7360e0f853f8 service glance] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:03.903264 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:03.904167 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:03.904444 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:03.904717 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] PATH_INFO: `/v3/users/c7fe59652eb24e17bfc4572a8b713f7a` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:03.905944 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7fe59652eb24e17bfc4572a8b713f7a)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:03.907852 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:03.955906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7fe59652eb24e17bfc4572a8b713f7a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:02:03.969856 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7fe59652eb24e17bfc4572a8b713f7a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:02:03.980374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7fe59652eb24e17bfc4572a8b713f7a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:02:03.987693 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7fe59652eb24e17bfc4572a8b713f7a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:02:03.988343 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7fe59652eb24e17bfc4572a8b713f7a'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:02:03.995181 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1378/2775] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:02:03 2022] DELETE /identity/v3/users/c7fe59652eb24e17bfc4572a8b713f7a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:02:03.999772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2b784-2b13-472a-8cf5-a06aa5d07c4f None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:04.000264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2b784-2b13-472a-8cf5-a06aa5d07c4f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:04.000462 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2b784-2b13-472a-8cf5-a06aa5d07c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:04.019049 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-daa2b784-2b13-472a-8cf5-a06aa5d07c4f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:02:04.020269 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-daa2b784-2b13-472a-8cf5-a06aa5d07c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:04.081219 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1398/2776] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:02:04.090934 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:04.092468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:04.196256 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0a5f6-2ea0-4e4f-8e63-c043caaaa128 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:04.204902 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee25e1-8f8c-468e-b4c9-e52c3615ee7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:04.206132 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee25e1-8f8c-468e-b4c9-e52c3615ee7b service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:04.206627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee25e1-8f8c-468e-b4c9-e52c3615ee7b service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:04.207094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee25e1-8f8c-468e-b4c9-e52c3615ee7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:04.218469 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee25e1-8f8c-468e-b4c9-e52c3615ee7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:04.220566 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee25e1-8f8c-468e-b4c9-e52c3615ee7b service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:04.234145 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1379/2777] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:02:04.552788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2b784-2b13-472a-8cf5-a06aa5d07c4f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:04.555032 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-daa2b784-2b13-472a-8cf5-a06aa5d07c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:04.567819 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2b784-2b13-472a-8cf5-a06aa5d07c4f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:04.574783 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:04.575806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:04.576284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:04.576564 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] PATH_INFO: `/v3/projects/299f1631bc9542aea0ae0362f6e32a89` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:04.577851 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=299f1631bc9542aea0ae0362f6e32a89)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:04.580283 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:04.635055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '299f1631bc9542aea0ae0362f6e32a89', 'request_id': 'req-0420109b-0bd5-4c74-88bb-773d31303f7a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:02:04.647260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '299f1631bc9542aea0ae0362f6e32a89', 'request_id': 'req-0420109b-0bd5-4c74-88bb-773d31303f7a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:02:04.655698 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '299f1631bc9542aea0ae0362f6e32a89', 'request_id': 'req-0420109b-0bd5-4c74-88bb-773d31303f7a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:02:04.670915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] The token cache is being invalidate because project 299f1631bc9542aea0ae0362f6e32a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:02:04.671206 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:02:04.672097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-0420109b-0bd5-4c74-88bb-773d31303f7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:02:04.674086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1399/2778] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:02:04 2022] DELETE /identity/v3/projects/299f1631bc9542aea0ae0362f6e32a89 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:02:04.680008 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fe988-1d6e-4c85-a132-8365506927fb None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:04.680369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fe988-1d6e-4c85-a132-8365506927fb None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:04.680764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fe988-1d6e-4c85-a132-8365506927fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:04.699618 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7a8fe988-1d6e-4c85-a132-8365506927fb None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:02:04.700832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7a8fe988-1d6e-4c85-a132-8365506927fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:04.724992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:02:04.770508 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1380/2779] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 20:02:04.778904 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff39ce9-faa0-4e13-9cd4-c5d9afe17baa None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:04.779193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff39ce9-faa0-4e13-9cd4-c5d9afe17baa None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:04.779478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff39ce9-faa0-4e13-9cd4-c5d9afe17baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:04.798278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-cff39ce9-faa0-4e13-9cd4-c5d9afe17baa None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:02:04.799498 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-cff39ce9-faa0-4e13-9cd4-c5d9afe17baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:04.808202 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1400/2780] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:02:04.812479 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fe988-1d6e-4c85-a132-8365506927fb None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:04.823061 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fe988-1d6e-4c85-a132-8365506927fb None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:04.828325 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb0320-07b4-4296-b5b2-dfb295000a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:04.828990 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb0320-07b4-4296-b5b2-dfb295000a47 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:04.829271 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb0320-07b4-4296-b5b2-dfb295000a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:04.829547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb0320-07b4-4296-b5b2-dfb295000a47 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:04.830309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb0320-07b4-4296-b5b2-dfb295000a47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:02:04.830620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb0320-07b4-4296-b5b2-dfb295000a47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:04.832221 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb0320-07b4-4296-b5b2-dfb295000a47 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:04.847991 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1381/2781] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 20:02:04.849121 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff39ce9-faa0-4e13-9cd4-c5d9afe17baa None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:04.859351 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff39ce9-faa0-4e13-9cd4-c5d9afe17baa None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:04.864151 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab579a-113e-4a10-a09f-8e80a7fd429d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:04.864811 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab579a-113e-4a10-a09f-8e80a7fd429d admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:04.865090 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab579a-113e-4a10-a09f-8e80a7fd429d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:04.865364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab579a-113e-4a10-a09f-8e80a7fd429d admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:04.866093 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab579a-113e-4a10-a09f-8e80a7fd429d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:04.867997 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab579a-113e-4a10-a09f-8e80a7fd429d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:04.887659 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:02:04.898051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1401/2782] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:02:04 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:02:04.902009 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb0320-07b4-4296-b5b2-dfb295000a47 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:04.913354 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb0320-07b4-4296-b5b2-dfb295000a47 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:04.972925 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820a22-11ed-4703-a571-f2baa16179a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:04.973614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820a22-11ed-4703-a571-f2baa16179a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:04.973912 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820a22-11ed-4703-a571-f2baa16179a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:04.974217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820a22-11ed-4703-a571-f2baa16179a0 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:04.975022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90820a22-11ed-4703-a571-f2baa16179a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:04.976936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90820a22-11ed-4703-a571-f2baa16179a0 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:04.998612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1382/2783] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:02:04 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:02:04.999862 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab579a-113e-4a10-a09f-8e80a7fd429d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:05.010836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab579a-113e-4a10-a09f-8e80a7fd429d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:05.015476 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311689e-5af3-4ed9-bb8a-1216e06ffe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:05.016128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311689e-5af3-4ed9-bb8a-1216e06ffe10 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:05.016661 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311689e-5af3-4ed9-bb8a-1216e06ffe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:05.016934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311689e-5af3-4ed9-bb8a-1216e06ffe10 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:05.017712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311689e-5af3-4ed9-bb8a-1216e06ffe10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:05.019302 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311689e-5af3-4ed9-bb8a-1216e06ffe10 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:05.030139 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1402/2784] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:02:05.034038 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820a22-11ed-4703-a571-f2baa16179a0 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:05.043814 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820a22-11ed-4703-a571-f2baa16179a0 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:05.048370 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b3039-a5e2-4e46-ae51-afc1a4581719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:05.049017 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b3039-a5e2-4e46-ae51-afc1a4581719 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:05.049300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b3039-a5e2-4e46-ae51-afc1a4581719 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:05.049598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b3039-a5e2-4e46-ae51-afc1a4581719 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:05.050330 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b3039-a5e2-4e46-ae51-afc1a4581719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:05.051956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b3039-a5e2-4e46-ae51-afc1a4581719 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:05.061839 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1383/2785] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:02:05.067172 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311689e-5af3-4ed9-bb8a-1216e06ffe10 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:05.077800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311689e-5af3-4ed9-bb8a-1216e06ffe10 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:05.082859 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac99f7-d513-48bb-a91a-5400a4a03894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:05.084545 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac99f7-d513-48bb-a91a-5400a4a03894 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:05.084855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac99f7-d513-48bb-a91a-5400a4a03894 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:05.085128 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac99f7-d513-48bb-a91a-5400a4a03894 admin admin] PATH_INFO: `/v3/projects/8fc7896ac2f04a87bbe1e19b92c87fc6/users/48d92050460c411997ab955ff07dc3cd/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:05.099776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac99f7-d513-48bb-a91a-5400a4a03894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc7896ac2f04a87bbe1e19b92c87fc6, user_id=48d92050460c411997ab955ff07dc3cd, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:05.102479 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac99f7-d513-48bb-a91a-5400a4a03894 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:05.115210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1403/2786] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:02:05 2022] PUT /identity/v3/projects/8fc7896ac2f04a87bbe1e19b92c87fc6/users/48d92050460c411997ab955ff07dc3cd/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:02:05.120302 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb548bdf-93a4-442d-af4b-c173dc2cbad5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:05.120591 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb548bdf-93a4-442d-af4b-c173dc2cbad5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:05.120879 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb548bdf-93a4-442d-af4b-c173dc2cbad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:05.138559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-fb548bdf-93a4-442d-af4b-c173dc2cbad5 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:02:05.139765 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fb548bdf-93a4-442d-af4b-c173dc2cbad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:05.199917 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1384/2787] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:02:05.210010 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac99f7-d513-48bb-a91a-5400a4a03894 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:05.211717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-13ac99f7-d513-48bb-a91a-5400a4a03894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:05.267717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac99f7-d513-48bb-a91a-5400a4a03894 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:05.271932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff934cb-7cd7-4700-b6a4-898fe6c7ddd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:05.272575 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff934cb-7cd7-4700-b6a4-898fe6c7ddd1 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:05.272856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff934cb-7cd7-4700-b6a4-898fe6c7ddd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:05.273132 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff934cb-7cd7-4700-b6a4-898fe6c7ddd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:05.282354 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff934cb-7cd7-4700-b6a4-898fe6c7ddd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:05.283786 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff934cb-7cd7-4700-b6a4-898fe6c7ddd1 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:05.294958 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1404/2788] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:02:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:02:06.458352 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b58023-d354-434f-b316-ca6c3130a253 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:06.458962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b58023-d354-434f-b316-ca6c3130a253 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:06.459208 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b58023-d354-434f-b316-ca6c3130a253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:06.491946 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-a2b58023-d354-434f-b316-ca6c3130a253 None None] MFA Rules not processed for user `48d92050460c411997ab955ff07dc3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:02:06.493898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a2b58023-d354-434f-b316-ca6c3130a253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:06.515277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:02:06.597015 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1385/2789] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:02:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:02:06.608525 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff934cb-7cd7-4700-b6a4-898fe6c7ddd1 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:06.610066 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-8ff934cb-7cd7-4700-b6a4-898fe6c7ddd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:06.672715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff934cb-7cd7-4700-b6a4-898fe6c7ddd1 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:06.676798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee0d9e-5a4d-4b64-8033-c637f9f4da00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:06.677775 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee0d9e-5a4d-4b64-8033-c637f9f4da00 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:06.677934 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee0d9e-5a4d-4b64-8033-c637f9f4da00 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:06.678210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee0d9e-5a4d-4b64-8033-c637f9f4da00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:06.687075 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee0d9e-5a4d-4b64-8033-c637f9f4da00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:06.688529 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee0d9e-5a4d-4b64-8033-c637f9f4da00 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:06.699159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1405/2790] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:02:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:02:08.032915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eca621-b1b1-4eb9-abc0-39c2015118c5 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:08.033417 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eca621-b1b1-4eb9-abc0-39c2015118c5 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:08.033524 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eca621-b1b1-4eb9-abc0-39c2015118c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:08.050848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-b8eca621-b1b1-4eb9-abc0-39c2015118c5 None None] MFA Rules not processed for user `48d92050460c411997ab955ff07dc3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:02:08.052070 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-b8eca621-b1b1-4eb9-abc0-39c2015118c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:08.058988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1386/2791] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:02:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:02:16.694081 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:02:18.045733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=22 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:02:44.617654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee0d9e-5a4d-4b64-8033-c637f9f4da00 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:44.619327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5bee0d9e-5a4d-4b64-8033-c637f9f4da00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:44.631993 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee0d9e-5a4d-4b64-8033-c637f9f4da00 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:44.636108 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91a3e5-845b-480c-8065-1f2a288306c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:44.636754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91a3e5-845b-480c-8065-1f2a288306c5 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:44.637043 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91a3e5-845b-480c-8065-1f2a288306c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:44.637312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91a3e5-845b-480c-8065-1f2a288306c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:44.639137 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5c91a3e5-845b-480c-8065-1f2a288306c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:44.715392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91a3e5-845b-480c-8065-1f2a288306c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:44.716829 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91a3e5-845b-480c-8065-1f2a288306c5 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:44.728654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1406/2792] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:02:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:02:45.910151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eca621-b1b1-4eb9-abc0-39c2015118c5 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:02:45.920299 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eca621-b1b1-4eb9-abc0-39c2015118c5 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:02:45.923863 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625ed6-aa09-43df-8df5-eaf5b1de2090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:02:45.924493 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625ed6-aa09-43df-8df5-eaf5b1de2090 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:02:45.924775 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625ed6-aa09-43df-8df5-eaf5b1de2090 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:02:45.925049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625ed6-aa09-43df-8df5-eaf5b1de2090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:02:45.927455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-20625ed6-aa09-43df-8df5-eaf5b1de2090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:02:45.941468 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20625ed6-aa09-43df-8df5-eaf5b1de2090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:02:45.942932 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20625ed6-aa09-43df-8df5-eaf5b1de2090 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:02:45.954066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1387/2793] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:02:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:02:54.722943 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=34 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:02:55.949214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=6 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:09.250355 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91a3e5-845b-480c-8065-1f2a288306c5 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:09.263100 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91a3e5-845b-480c-8065-1f2a288306c5 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:09.266746 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8f644-21c1-4ea1-a456-b1045934a280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:09.267380 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8f644-21c1-4ea1-a456-b1045934a280 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:09.267915 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8f644-21c1-4ea1-a456-b1045934a280 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:09.268190 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8f644-21c1-4ea1-a456-b1045934a280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:09.270051 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-2aa8f644-21c1-4ea1-a456-b1045934a280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:09.283046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8f644-21c1-4ea1-a456-b1045934a280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:09.284459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8f644-21c1-4ea1-a456-b1045934a280 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:09.294452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1407/2794] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:16.657147 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625ed6-aa09-43df-8df5-eaf5b1de2090 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:16.658635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-20625ed6-aa09-43df-8df5-eaf5b1de2090 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:16.672273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625ed6-aa09-43df-8df5-eaf5b1de2090 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:16.676726 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:16.677614 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:16.677918 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:16.678190 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] PATH_INFO: `/v3/users/80267c31f3fc4b3fbc8dc7ed03326eeb` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:16.679399 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80267c31f3fc4b3fbc8dc7ed03326eeb)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:16.681637 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:16.727154 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80267c31f3fc4b3fbc8dc7ed03326eeb'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:16.738950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80267c31f3fc4b3fbc8dc7ed03326eeb'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:16.746329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80267c31f3fc4b3fbc8dc7ed03326eeb'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:16.753527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80267c31f3fc4b3fbc8dc7ed03326eeb'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:16.754196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80267c31f3fc4b3fbc8dc7ed03326eeb'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:16.760684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1388/2795] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:03:16 2022] DELETE /identity/v3/users/80267c31f3fc4b3fbc8dc7ed03326eeb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:16.939200 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8f644-21c1-4ea1-a456-b1045934a280 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:16.951427 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8f644-21c1-4ea1-a456-b1045934a280 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:17.014788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:17.015569 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:17.015847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:17.016119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] PATH_INFO: `/v3/projects/262e5b1a6f6248cca455437d23b8d441` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:17.017292 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=262e5b1a6f6248cca455437d23b8d441)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:17.019527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:17.065113 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '262e5b1a6f6248cca455437d23b8d441', 'request_id': 'req-5d9b0fbc-420d-4b7e-adca-73238dc128f2'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.072839 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '262e5b1a6f6248cca455437d23b8d441', 'request_id': 'req-5d9b0fbc-420d-4b7e-adca-73238dc128f2'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.078949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '262e5b1a6f6248cca455437d23b8d441', 'request_id': 'req-5d9b0fbc-420d-4b7e-adca-73238dc128f2'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.089131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] The token cache is being invalidate because project 262e5b1a6f6248cca455437d23b8d441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:03:17.089414 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.090067 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.091602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1408/2796] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:03:16 2022] DELETE /identity/v3/projects/262e5b1a6f6248cca455437d23b8d441 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:17.095562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:17.096966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:17.114140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875419-79c6-4048-ad7c-6970fa8c39d2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:17.171835 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:17.172710 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:17.172996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:17.173273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] PATH_INFO: `/v3/users/b771b0dd79db41c4ab6ab0ce9037bac9` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:17.174489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b771b0dd79db41c4ab6ab0ce9037bac9)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:17.176414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:17.222267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b771b0dd79db41c4ab6ab0ce9037bac9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.234787 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b771b0dd79db41c4ab6ab0ce9037bac9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.242136 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b771b0dd79db41c4ab6ab0ce9037bac9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.244885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:17.253135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b771b0dd79db41c4ab6ab0ce9037bac9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.254059 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b771b0dd79db41c4ab6ab0ce9037bac9'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.260320 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1389/2797] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:03:17 2022] DELETE /identity/v3/users/b771b0dd79db41c4ab6ab0ce9037bac9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:17.454587 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:17.458252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:17.476146 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b0fbc-420d-4b7e-adca-73238dc128f2 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:17.562490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:17.563261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:17.563540 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:17.563807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] PATH_INFO: `/v3/projects/bdfabef2ed454d6590af05df16f55610` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:17.565467 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdfabef2ed454d6590af05df16f55610)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:17.567547 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:17.591443 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:17.621640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdfabef2ed454d6590af05df16f55610', 'request_id': 'req-253ef714-ff10-48d2-8aeb-f1facbdda80a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.629523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdfabef2ed454d6590af05df16f55610', 'request_id': 'req-253ef714-ff10-48d2-8aeb-f1facbdda80a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.636663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdfabef2ed454d6590af05df16f55610', 'request_id': 'req-253ef714-ff10-48d2-8aeb-f1facbdda80a'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.647492 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] The token cache is being invalidate because project bdfabef2ed454d6590af05df16f55610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:03:17.647779 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.648438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:17.650863 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1409/2798] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:03:17 2022] DELETE /identity/v3/projects/bdfabef2ed454d6590af05df16f55610 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:19.797305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:19.798733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:19.856783 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be15ff-f246-471e-8892-0b8562d69f5d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:19.904111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975f922-88f3-4f0b-9b34-6bfeab376cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:19.904745 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f922-88f3-4f0b-9b34-6bfeab376cac service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:19.905023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f922-88f3-4f0b-9b34-6bfeab376cac service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:19.905296 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f922-88f3-4f0b-9b34-6bfeab376cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:19.907053 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d975f922-88f3-4f0b-9b34-6bfeab376cac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:19.966314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975f922-88f3-4f0b-9b34-6bfeab376cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:19.967731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975f922-88f3-4f0b-9b34-6bfeab376cac service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:19.976422 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1390/2799] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:20.073317 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:20.074766 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:20.131117 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ef714-ff10-48d2-8aeb-f1facbdda80a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:20.135068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e223d4-823a-4332-bdd9-0bef00048016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:20.135701 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e223d4-823a-4332-bdd9-0bef00048016 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:20.135978 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e223d4-823a-4332-bdd9-0bef00048016 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:20.136249 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e223d4-823a-4332-bdd9-0bef00048016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:20.138101 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-47e223d4-823a-4332-bdd9-0bef00048016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:20.200753 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e223d4-823a-4332-bdd9-0bef00048016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:20.202221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e223d4-823a-4332-bdd9-0bef00048016 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:20.211591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1410/2800] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:22.069066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440003b1-dcd1-41cf-a9ec-225c864c6e2d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:22.069620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440003b1-dcd1-41cf-a9ec-225c864c6e2d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:22.069620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440003b1-dcd1-41cf-a9ec-225c864c6e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:22.090323 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-440003b1-dcd1-41cf-a9ec-225c864c6e2d None None] MFA Rules not processed for user `48d92050460c411997ab955ff07dc3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:22.091553 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-440003b1-dcd1-41cf-a9ec-225c864c6e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:22.150314 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1391/2801] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:22.469173 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e223d4-823a-4332-bdd9-0bef00048016 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:22.480805 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e223d4-823a-4332-bdd9-0bef00048016 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:22.485230 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d77cc-2c18-4ce7-b189-6d66693fdf80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:22.486231 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d77cc-2c18-4ce7-b189-6d66693fdf80 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:22.486730 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d77cc-2c18-4ce7-b189-6d66693fdf80 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:22.487214 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d77cc-2c18-4ce7-b189-6d66693fdf80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:22.489527 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7a7d77cc-2c18-4ce7-b189-6d66693fdf80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:22.508482 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d77cc-2c18-4ce7-b189-6d66693fdf80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:22.510806 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d77cc-2c18-4ce7-b189-6d66693fdf80 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:22.519585 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1411/2802] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:23.044577 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440003b1-dcd1-41cf-a9ec-225c864c6e2d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:23.047247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-440003b1-dcd1-41cf-a9ec-225c864c6e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:23.059617 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=100 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:23.136099 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440003b1-dcd1-41cf-a9ec-225c864c6e2d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:23.144407 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc48ddf-9224-4407-a211-52af4f298b70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:23.145424 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc48ddf-9224-4407-a211-52af4f298b70 service placement] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:23.145761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc48ddf-9224-4407-a211-52af4f298b70 service placement] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:23.146020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc48ddf-9224-4407-a211-52af4f298b70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:23.147915 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-0fc48ddf-9224-4407-a211-52af4f298b70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:23.161569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc48ddf-9224-4407-a211-52af4f298b70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:23.163351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc48ddf-9224-4407-a211-52af4f298b70 service placement] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:23.188549 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1392/2803] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:23.265649 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d77cc-2c18-4ce7-b189-6d66693fdf80 service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:23.267856 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7a7d77cc-2c18-4ce7-b189-6d66693fdf80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:23.292554 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d77cc-2c18-4ce7-b189-6d66693fdf80 service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:23.299628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:23.300430 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:23.300711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:23.300983 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] PATH_INFO: `/v3/users/8f2b9a6934d34f58a09a4cefb154371c` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:23.302348 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f2b9a6934d34f58a09a4cefb154371c)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:23.304307 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:23.357406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:23.364669 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f2b9a6934d34f58a09a4cefb154371c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:23.380238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f2b9a6934d34f58a09a4cefb154371c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:23.380868 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f2b9a6934d34f58a09a4cefb154371c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:23.393927 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f2b9a6934d34f58a09a4cefb154371c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:23.394635 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-962d7e87-0a6c-4928-ba59-3ea261fa07fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f2b9a6934d34f58a09a4cefb154371c'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:23.413419 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1412/2804] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:03:23 2022] DELETE /identity/v3/users/8f2b9a6934d34f58a09a4cefb154371c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:23.625196 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc48ddf-9224-4407-a211-52af4f298b70 service placement] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:23.647034 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc48ddf-9224-4407-a211-52af4f298b70 service placement] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:23.706548 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:23.707413 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:23.707695 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:23.707970 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] PATH_INFO: `/v3/projects/983da191c3d44e608e40f289e26d3603` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:23.709140 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=983da191c3d44e608e40f289e26d3603)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:23.711081 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:23.757076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983da191c3d44e608e40f289e26d3603', 'request_id': 'req-490d6dc2-5649-4767-9b67-9a0045342d0c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:23.763684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983da191c3d44e608e40f289e26d3603', 'request_id': 'req-490d6dc2-5649-4767-9b67-9a0045342d0c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:23.769971 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983da191c3d44e608e40f289e26d3603', 'request_id': 'req-490d6dc2-5649-4767-9b67-9a0045342d0c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:23.781536 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] The token cache is being invalidate because project 983da191c3d44e608e40f289e26d3603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:03:23.781859 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:23.782504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-490d6dc2-5649-4767-9b67-9a0045342d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:23.784184 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1393/2805] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:03:23 2022] DELETE /identity/v3/projects/983da191c3d44e608e40f289e26d3603 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:23.789377 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3db58a-8024-4aac-8607-1b9ab2cc650f None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:23.789883 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3db58a-8024-4aac-8607-1b9ab2cc650f None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:23.790055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3db58a-8024-4aac-8607-1b9ab2cc650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:23.808949 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-9d3db58a-8024-4aac-8607-1b9ab2cc650f None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:23.810134 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-9d3db58a-8024-4aac-8607-1b9ab2cc650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:23.868225 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1413/2806] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 20:03:23.876172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d6005-a5de-45a8-8d5c-705d0d9dd7c1 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:23.876684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d6005-a5de-45a8-8d5c-705d0d9dd7c1 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:23.876961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d6005-a5de-45a8-8d5c-705d0d9dd7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:23.897434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-518d6005-a5de-45a8-8d5c-705d0d9dd7c1 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:23.898743 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-518d6005-a5de-45a8-8d5c-705d0d9dd7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:23.907410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1394/2807] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:23.911641 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3db58a-8024-4aac-8607-1b9ab2cc650f None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:23.923003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3db58a-8024-4aac-8607-1b9ab2cc650f None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:23.927670 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea395fa-074c-4ca9-8d94-44bb207c321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:23.928847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea395fa-074c-4ca9-8d94-44bb207c321a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:23.928847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea395fa-074c-4ca9-8d94-44bb207c321a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:23.928847 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea395fa-074c-4ca9-8d94-44bb207c321a admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:23.929602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea395fa-074c-4ca9-8d94-44bb207c321a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:03:23.929920 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea395fa-074c-4ca9-8d94-44bb207c321a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:23.931865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea395fa-074c-4ca9-8d94-44bb207c321a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:23.948483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1414/2808] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 20:03:23.949796 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d6005-a5de-45a8-8d5c-705d0d9dd7c1 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:23.959996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d6005-a5de-45a8-8d5c-705d0d9dd7c1 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:23.984898 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99212e6-bf1c-460d-8949-0cee91d71518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:23.985570 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99212e6-bf1c-460d-8949-0cee91d71518 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:23.985881 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99212e6-bf1c-460d-8949-0cee91d71518 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:23.986152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99212e6-bf1c-460d-8949-0cee91d71518 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:23.986849 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99212e6-bf1c-460d-8949-0cee91d71518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:23.988753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99212e6-bf1c-460d-8949-0cee91d71518 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:24.016223 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:24.028993 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1395/2809] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:03:23 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:03:24.030364 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea395fa-074c-4ca9-8d94-44bb207c321a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:24.043278 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea395fa-074c-4ca9-8d94-44bb207c321a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:24.117215 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e36186-9de1-41b9-a819-e28f2ce5e3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:24.117916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e36186-9de1-41b9-a819-e28f2ce5e3e7 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:24.118189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e36186-9de1-41b9-a819-e28f2ce5e3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:24.118530 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e36186-9de1-41b9-a819-e28f2ce5e3e7 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:24.119849 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e36186-9de1-41b9-a819-e28f2ce5e3e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:24.121846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e36186-9de1-41b9-a819-e28f2ce5e3e7 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:24.146350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1415/2810] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:03:24 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:03:24.147342 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99212e6-bf1c-460d-8949-0cee91d71518 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:24.157777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99212e6-bf1c-460d-8949-0cee91d71518 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:24.162534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6bcb6-6df8-4b8e-a59d-19fa2fb15736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:24.163193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6bcb6-6df8-4b8e-a59d-19fa2fb15736 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:24.163472 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6bcb6-6df8-4b8e-a59d-19fa2fb15736 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:24.163741 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6bcb6-6df8-4b8e-a59d-19fa2fb15736 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:24.164465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6bcb6-6df8-4b8e-a59d-19fa2fb15736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:24.166682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6bcb6-6df8-4b8e-a59d-19fa2fb15736 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:24.177234 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1396/2811] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 20:03:24.181833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e36186-9de1-41b9-a819-e28f2ce5e3e7 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:24.191853 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e36186-9de1-41b9-a819-e28f2ce5e3e7 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:24.196417 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cb425-df31-4ad0-bc3e-b76c14af50d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:24.197305 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cb425-df31-4ad0-bc3e-b76c14af50d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:24.197598 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cb425-df31-4ad0-bc3e-b76c14af50d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:24.197882 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cb425-df31-4ad0-bc3e-b76c14af50d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:24.198599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cb425-df31-4ad0-bc3e-b76c14af50d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:24.200183 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cb425-df31-4ad0-bc3e-b76c14af50d6 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:24.210189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1416/2812] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:24.215186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6bcb6-6df8-4b8e-a59d-19fa2fb15736 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:24.225169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6bcb6-6df8-4b8e-a59d-19fa2fb15736 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:24.230996 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f1f02-264d-4396-8ce7-3b7108caeeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:24.231906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f1f02-264d-4396-8ce7-3b7108caeeb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:24.232530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f1f02-264d-4396-8ce7-3b7108caeeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:24.232530 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f1f02-264d-4396-8ce7-3b7108caeeb8 admin admin] PATH_INFO: `/v3/projects/5b13ed2b118641f286de3f0c3696bcb1/users/31a2f17f0fd14bf88e481af0575940f8/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:24.248764 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f1f02-264d-4396-8ce7-3b7108caeeb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b13ed2b118641f286de3f0c3696bcb1, user_id=31a2f17f0fd14bf88e481af0575940f8, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:24.251447 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f1f02-264d-4396-8ce7-3b7108caeeb8 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:24.265868 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1397/2813] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:03:24 2022] PUT /identity/v3/projects/5b13ed2b118641f286de3f0c3696bcb1/users/31a2f17f0fd14bf88e481af0575940f8/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:24.277992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9354a-8628-47a6-98cb-27346adf421e None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:24.278285 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9354a-8628-47a6-98cb-27346adf421e None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:24.278596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9354a-8628-47a6-98cb-27346adf421e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:24.306646 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-f5c9354a-8628-47a6-98cb-27346adf421e None None] MFA Rules not processed for user `31a2f17f0fd14bf88e481af0575940f8`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:24.307837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f5c9354a-8628-47a6-98cb-27346adf421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:24.318791 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:24.377957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1417/2814] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:24.389248 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f1f02-264d-4396-8ce7-3b7108caeeb8 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:24.391148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-eb1f1f02-264d-4396-8ce7-3b7108caeeb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:24.471711 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f1f02-264d-4396-8ce7-3b7108caeeb8 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:24.477291 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ffc9e-2534-46c7-9a80-7927fe17ac28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:24.478788 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ffc9e-2534-46c7-9a80-7927fe17ac28 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:24.479086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ffc9e-2534-46c7-9a80-7927fe17ac28 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:24.479368 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ffc9e-2534-46c7-9a80-7927fe17ac28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:24.487555 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ffc9e-2534-46c7-9a80-7927fe17ac28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:24.489153 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ffc9e-2534-46c7-9a80-7927fe17ac28 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:24.499478 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1398/2815] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:25.747973 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9354a-8628-47a6-98cb-27346adf421e None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:25.749546 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-f5c9354a-8628-47a6-98cb-27346adf421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:25.807217 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9354a-8628-47a6-98cb-27346adf421e None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:25.810800 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad947fd-83ca-4cda-9c28-56b9c9da0b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:25.811441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad947fd-83ca-4cda-9c28-56b9c9da0b27 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:25.811724 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad947fd-83ca-4cda-9c28-56b9c9da0b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:25.812022 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad947fd-83ca-4cda-9c28-56b9c9da0b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:25.813855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7ad947fd-83ca-4cda-9c28-56b9c9da0b27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:25.823599 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad947fd-83ca-4cda-9c28-56b9c9da0b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:25.825086 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad947fd-83ca-4cda-9c28-56b9c9da0b27 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:25.833374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1418/2816] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:27.220241 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ffc9e-2534-46c7-9a80-7927fe17ac28 service nova] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:27.286259 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ffc9e-2534-46c7-9a80-7927fe17ac28 service nova] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:27.292809 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:27.293794 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:27.294217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:27.294559 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] PATH_INFO: `/v3/users/31a2f17f0fd14bf88e481af0575940f8` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:27.296369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a2f17f0fd14bf88e481af0575940f8)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:27.299063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:27.368063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a2f17f0fd14bf88e481af0575940f8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:27.371813 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:27.385194 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a2f17f0fd14bf88e481af0575940f8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:27.394270 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a2f17f0fd14bf88e481af0575940f8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:27.403013 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a2f17f0fd14bf88e481af0575940f8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:27.403727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a2f17f0fd14bf88e481af0575940f8'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:27.410867 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1399/2817] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:03:27 2022] DELETE /identity/v3/users/31a2f17f0fd14bf88e481af0575940f8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:27.416068 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2ad9d-288e-48de-87e4-c9d8326c1d1a None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:27.416638 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2ad9d-288e-48de-87e4-c9d8326c1d1a None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:27.416776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2ad9d-288e-48de-87e4-c9d8326c1d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:27.435558 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-b4f2ad9d-288e-48de-87e4-c9d8326c1d1a None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:27.436743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b4f2ad9d-288e-48de-87e4-c9d8326c1d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:27.496450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1419/2818] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:27.506165 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:27.507585 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:27.567380 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30b375-ea2d-4296-bc55-8bb0ff04db4a admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:27.571043 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34d97e-a3c9-4f1a-9d0a-1ea57dfdc5e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:27.571712 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34d97e-a3c9-4f1a-9d0a-1ea57dfdc5e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:27.571992 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34d97e-a3c9-4f1a-9d0a-1ea57dfdc5e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:27.572268 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34d97e-a3c9-4f1a-9d0a-1ea57dfdc5e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:27.581267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34d97e-a3c9-4f1a-9d0a-1ea57dfdc5e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:27.582727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34d97e-a3c9-4f1a-9d0a-1ea57dfdc5e6 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:27.592883 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1400/2819] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 20:03:27.866252 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2ad9d-288e-48de-87e4-c9d8326c1d1a None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:27.867683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-b4f2ad9d-288e-48de-87e4-c9d8326c1d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:27.880400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2ad9d-288e-48de-87e4-c9d8326c1d1a None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:27.885003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:27.885911 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:27.886189 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:27.886452 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] PATH_INFO: `/v3/projects/5b13ed2b118641f286de3f0c3696bcb1` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:27.887628 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b13ed2b118641f286de3f0c3696bcb1)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:27.889564 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:27.937425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b13ed2b118641f286de3f0c3696bcb1', 'request_id': 'req-ab51784d-5dbb-449b-ae6f-226586be670b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:27.939211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:27.945435 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b13ed2b118641f286de3f0c3696bcb1', 'request_id': 'req-ab51784d-5dbb-449b-ae6f-226586be670b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:27.952115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b13ed2b118641f286de3f0c3696bcb1', 'request_id': 'req-ab51784d-5dbb-449b-ae6f-226586be670b'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:27.962391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] The token cache is being invalidate because project 5b13ed2b118641f286de3f0c3696bcb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:03:27.962695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:27.963318 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-ab51784d-5dbb-449b-ae6f-226586be670b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:27.964875 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1420/2820] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:03:27 2022] DELETE /identity/v3/projects/5b13ed2b118641f286de3f0c3696bcb1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:27.970058 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9fdae-87fc-4d97-96ba-04e2502a4ced None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:27.970348 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9fdae-87fc-4d97-96ba-04e2502a4ced None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:27.970627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9fdae-87fc-4d97-96ba-04e2502a4ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:27.988654 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-afd9fdae-87fc-4d97-96ba-04e2502a4ced None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:27.989886 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-afd9fdae-87fc-4d97-96ba-04e2502a4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:28.047563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1401/2821] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:28.055052 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b661a30-9e6a-4810-9ea8-e41661293aec None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:28.055342 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b661a30-9e6a-4810-9ea8-e41661293aec None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:28.055626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b661a30-9e6a-4810-9ea8-e41661293aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:28.074122 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-3b661a30-9e6a-4810-9ea8-e41661293aec None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:28.075326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3b661a30-9e6a-4810-9ea8-e41661293aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:28.083240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1421/2822] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:28.088455 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9fdae-87fc-4d97-96ba-04e2502a4ced None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:28.098023 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9fdae-87fc-4d97-96ba-04e2502a4ced None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:28.102547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f1196-8984-46f5-bf79-462b65c08b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:28.104329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f1196-8984-46f5-bf79-462b65c08b69 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:28.104605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f1196-8984-46f5-bf79-462b65c08b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:28.104877 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f1196-8984-46f5-bf79-462b65c08b69 admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:28.105623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f1196-8984-46f5-bf79-462b65c08b69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:03:28.105936 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f1196-8984-46f5-bf79-462b65c08b69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:28.107534 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f1196-8984-46f5-bf79-462b65c08b69 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:28.119531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1402/2823] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 20:03:28.123770 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b661a30-9e6a-4810-9ea8-e41661293aec None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:28.132837 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b661a30-9e6a-4810-9ea8-e41661293aec None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:28.137719 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8412-a910-4d52-87e1-6b464f143648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:28.138253 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8412-a910-4d52-87e1-6b464f143648 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:28.138534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8412-a910-4d52-87e1-6b464f143648 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:28.138810 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8412-a910-4d52-87e1-6b464f143648 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:28.139496 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec8412-a910-4d52-87e1-6b464f143648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:28.141425 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec8412-a910-4d52-87e1-6b464f143648 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:28.167742 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1422/2824] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:03:28 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:03:28.172069 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f1196-8984-46f5-bf79-462b65c08b69 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:28.182138 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f1196-8984-46f5-bf79-462b65c08b69 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:28.232018 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:28.238539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3b46f-4c77-4fef-bb93-2360ad967328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:28.239209 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3b46f-4c77-4fef-bb93-2360ad967328 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:28.239782 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3b46f-4c77-4fef-bb93-2360ad967328 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:28.240120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3b46f-4c77-4fef-bb93-2360ad967328 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:28.240895 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3b46f-4c77-4fef-bb93-2360ad967328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:28.243285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3b46f-4c77-4fef-bb93-2360ad967328 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:28.263945 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1403/2825] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:03:28 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:03:28.268148 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8412-a910-4d52-87e1-6b464f143648 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:28.279386 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8412-a910-4d52-87e1-6b464f143648 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:28.283932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf9275-9b4b-4151-8ec4-0e616099ecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:28.284573 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf9275-9b4b-4151-8ec4-0e616099ecd0 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:28.284850 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf9275-9b4b-4151-8ec4-0e616099ecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:28.285120 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf9275-9b4b-4151-8ec4-0e616099ecd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:28.285869 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf9275-9b4b-4151-8ec4-0e616099ecd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:28.287441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf9275-9b4b-4151-8ec4-0e616099ecd0 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:28.298212 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1423/2826] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:28.302062 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3b46f-4c77-4fef-bb93-2360ad967328 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:28.312966 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3b46f-4c77-4fef-bb93-2360ad967328 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:28.318623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d6e5e-e997-4c1d-add6-d1b35e92465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:28.319277 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d6e5e-e997-4c1d-add6-d1b35e92465b admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:28.319558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d6e5e-e997-4c1d-add6-d1b35e92465b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:28.319833 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d6e5e-e997-4c1d-add6-d1b35e92465b admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:28.320547 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d6e5e-e997-4c1d-add6-d1b35e92465b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:28.322808 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d6e5e-e997-4c1d-add6-d1b35e92465b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:28.332977 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1404/2827] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:28.336985 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf9275-9b4b-4151-8ec4-0e616099ecd0 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:28.347280 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf9275-9b4b-4151-8ec4-0e616099ecd0 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:28.351798 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e374b-9deb-4331-81e4-b5d79b0972ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:28.352668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e374b-9deb-4331-81e4-b5d79b0972ee admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:28.353493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e374b-9deb-4331-81e4-b5d79b0972ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:28.353784 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e374b-9deb-4331-81e4-b5d79b0972ee admin admin] PATH_INFO: `/v3/projects/91265593821f4a1c8bc1f76aba94e493/users/f1ca3d27f4444d62a42621d5490ec09f/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:28.367761 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e374b-9deb-4331-81e4-b5d79b0972ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=91265593821f4a1c8bc1f76aba94e493, user_id=f1ca3d27f4444d62a42621d5490ec09f, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:28.370566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e374b-9deb-4331-81e4-b5d79b0972ee admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:28.382712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1424/2828] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:03:28 2022] PUT /identity/v3/projects/91265593821f4a1c8bc1f76aba94e493/users/f1ca3d27f4444d62a42621d5490ec09f/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:28.393056 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33d2c6-fec5-4551-9e40-ff0172051957 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:28.393636 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33d2c6-fec5-4551-9e40-ff0172051957 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:28.393947 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33d2c6-fec5-4551-9e40-ff0172051957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:28.423500 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-2d33d2c6-fec5-4551-9e40-ff0172051957 None None] MFA Rules not processed for user `f1ca3d27f4444d62a42621d5490ec09f`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:28.424753 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2d33d2c6-fec5-4551-9e40-ff0172051957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:28.490503 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1405/2829] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:28.501999 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e374b-9deb-4331-81e4-b5d79b0972ee admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:28.503899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-964e374b-9deb-4331-81e4-b5d79b0972ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:28.567193 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e374b-9deb-4331-81e4-b5d79b0972ee admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:28.570852 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0647f58-3514-4281-887e-028f1b669a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:28.572019 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647f58-3514-4281-887e-028f1b669a96 service cinder] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:28.572594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647f58-3514-4281-887e-028f1b669a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:28.572594 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0647f58-3514-4281-887e-028f1b669a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:28.581552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0647f58-3514-4281-887e-028f1b669a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:28.583092 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0647f58-3514-4281-887e-028f1b669a96 service cinder] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:28.594300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1425/2830] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:30.557239 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33d2c6-fec5-4551-9e40-ff0172051957 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:30.558777 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-2d33d2c6-fec5-4551-9e40-ff0172051957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:30.634504 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33d2c6-fec5-4551-9e40-ff0172051957 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:30.639958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a809b3-c05f-4b4c-bcd6-dce30827f685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:30.640598 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a809b3-c05f-4b4c-bcd6-dce30827f685 service nova] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:30.640879 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a809b3-c05f-4b4c-bcd6-dce30827f685 service nova] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:30.641152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a809b3-c05f-4b4c-bcd6-dce30827f685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:30.642935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-08a809b3-c05f-4b4c-bcd6-dce30827f685 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:30.655594 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a809b3-c05f-4b4c-bcd6-dce30827f685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:30.657169 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a809b3-c05f-4b4c-bcd6-dce30827f685 service nova] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:30.668195 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1406/2831] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:36.919965 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0647f58-3514-4281-887e-028f1b669a96 service cinder] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:36.986750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0647f58-3514-4281-887e-028f1b669a96 service cinder] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:36.991552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:36.992471 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:36.992752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:36.993029 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] PATH_INFO: `/v3/users/f1ca3d27f4444d62a42621d5490ec09f` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:36.994290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ca3d27f4444d62a42621d5490ec09f)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:36.997021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:37.019597 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:INSERT=3,keystone:DELETE=5,keystone:SELECT=92 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:37.045327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ca3d27f4444d62a42621d5490ec09f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:37.052483 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ca3d27f4444d62a42621d5490ec09f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:37.053111 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ca3d27f4444d62a42621d5490ec09f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:37.065114 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ca3d27f4444d62a42621d5490ec09f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:37.065776 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ca3d27f4444d62a42621d5490ec09f'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:37.078149 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1426/2832] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:03:36 2022] DELETE /identity/v3/users/f1ca3d27f4444d62a42621d5490ec09f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:37.082864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3942f-55e9-4016-ae89-ff7d8ca743e9 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:37.083350 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3942f-55e9-4016-ae89-ff7d8ca743e9 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:37.083457 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3942f-55e9-4016-ae89-ff7d8ca743e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:37.102281 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c1f3942f-55e9-4016-ae89-ff7d8ca743e9 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:37.104388 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c1f3942f-55e9-4016-ae89-ff7d8ca743e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:37.147862 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:37.165237 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1407/2833] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:37.175438 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:37.176839 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:37.233855 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788904aa-58cf-4fcc-bfd3-6dbf9a802c49 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:37.237441 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024c0f6-3aa7-430a-bd3a-0eaac931dd13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:37.238107 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024c0f6-3aa7-430a-bd3a-0eaac931dd13 service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:37.238396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024c0f6-3aa7-430a-bd3a-0eaac931dd13 service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:37.238668 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024c0f6-3aa7-430a-bd3a-0eaac931dd13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:37.256246 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024c0f6-3aa7-430a-bd3a-0eaac931dd13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:37.257712 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024c0f6-3aa7-430a-bd3a-0eaac931dd13 service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:37.267626 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1427/2834] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:37.550121 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3942f-55e9-4016-ae89-ff7d8ca743e9 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:37.551805 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c1f3942f-55e9-4016-ae89-ff7d8ca743e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:37.565482 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3942f-55e9-4016-ae89-ff7d8ca743e9 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:37.570152 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:37.571088 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:37.571373 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:37.571645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] PATH_INFO: `/v3/projects/91265593821f4a1c8bc1f76aba94e493` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:37.573122 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91265593821f4a1c8bc1f76aba94e493)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:37.574834 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:37.621096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91265593821f4a1c8bc1f76aba94e493', 'request_id': 'req-1d5f825b-9f3e-4807-9185-04e09d178c42'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:37.628443 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91265593821f4a1c8bc1f76aba94e493', 'request_id': 'req-1d5f825b-9f3e-4807-9185-04e09d178c42'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:37.634004 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91265593821f4a1c8bc1f76aba94e493', 'request_id': 'req-1d5f825b-9f3e-4807-9185-04e09d178c42'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:37.645077 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] The token cache is being invalidate because project 91265593821f4a1c8bc1f76aba94e493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:03:37.645364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:37.646033 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-1d5f825b-9f3e-4807-9185-04e09d178c42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:37.647571 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1408/2835] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:03:37 2022] DELETE /identity/v3/projects/91265593821f4a1c8bc1f76aba94e493 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:37.653133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4ab76-502f-452a-abbc-128c3221fcf0 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:37.653423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4ab76-502f-452a-abbc-128c3221fcf0 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:37.653741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4ab76-502f-452a-abbc-128c3221fcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:37.672184 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-50a4ab76-502f-452a-abbc-128c3221fcf0 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:37.673329 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-50a4ab76-502f-452a-abbc-128c3221fcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:37.732065 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1428/2836] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:37.741007 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf544c-7db0-4cb1-bdfc-fdc1578c296d None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:37.741295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf544c-7db0-4cb1-bdfc-fdc1578c296d None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:37.741601 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf544c-7db0-4cb1-bdfc-fdc1578c296d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:37.760048 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-d1bf544c-7db0-4cb1-bdfc-fdc1578c296d None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:37.761230 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-d1bf544c-7db0-4cb1-bdfc-fdc1578c296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:37.769445 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1409/2837] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:37.773561 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4ab76-502f-452a-abbc-128c3221fcf0 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:37.783409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4ab76-502f-452a-abbc-128c3221fcf0 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:37.787966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a8e49-251c-4fd4-97f3-c17f7ddd341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:37.788648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a8e49-251c-4fd4-97f3-c17f7ddd341d admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:37.788932 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a8e49-251c-4fd4-97f3-c17f7ddd341d admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:37.789210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a8e49-251c-4fd4-97f3-c17f7ddd341d admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:37.789959 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a8e49-251c-4fd4-97f3-c17f7ddd341d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:03:37.790267 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a8e49-251c-4fd4-97f3-c17f7ddd341d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:37.792502 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a8e49-251c-4fd4-97f3-c17f7ddd341d admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:37.804003 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1429/2838] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 20:03:37.809091 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf544c-7db0-4cb1-bdfc-fdc1578c296d None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:37.818251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf544c-7db0-4cb1-bdfc-fdc1578c296d None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:37.823151 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d626f-766c-4e5b-b9b2-654ddedf98e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:37.823789 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d626f-766c-4e5b-b9b2-654ddedf98e7 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:37.824078 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d626f-766c-4e5b-b9b2-654ddedf98e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:37.824341 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d626f-766c-4e5b-b9b2-654ddedf98e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:37.825030 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d626f-766c-4e5b-b9b2-654ddedf98e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:37.827086 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d626f-766c-4e5b-b9b2-654ddedf98e7 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:37.853514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1410/2839] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:03:37 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:03:37.858204 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a8e49-251c-4fd4-97f3-c17f7ddd341d admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:37.868112 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a8e49-251c-4fd4-97f3-c17f7ddd341d admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:37.892874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:37.928313 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579ea73-9964-44eb-b3b2-7e24ed57f615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:37.928969 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579ea73-9964-44eb-b3b2-7e24ed57f615 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:37.929179 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579ea73-9964-44eb-b3b2-7e24ed57f615 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:37.929454 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579ea73-9964-44eb-b3b2-7e24ed57f615 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:37.930185 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579ea73-9964-44eb-b3b2-7e24ed57f615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:37.932729 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579ea73-9964-44eb-b3b2-7e24ed57f615 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:37.953767 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1430/2840] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:03:37 2022] POST /identity/v3/users => generated 303 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:03:37.957927 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d626f-766c-4e5b-b9b2-654ddedf98e7 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:37.968448 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d626f-766c-4e5b-b9b2-654ddedf98e7 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:37.973046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a74ea9-0e8b-4b6f-9d2f-abca5f378612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:37.973740 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a74ea9-0e8b-4b6f-9d2f-abca5f378612 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:37.974022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a74ea9-0e8b-4b6f-9d2f-abca5f378612 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:37.974295 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a74ea9-0e8b-4b6f-9d2f-abca5f378612 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:37.975568 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a74ea9-0e8b-4b6f-9d2f-abca5f378612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:37.977164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a74ea9-0e8b-4b6f-9d2f-abca5f378612 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:37.988276 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1411/2841] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:37.992274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579ea73-9964-44eb-b3b2-7e24ed57f615 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:38.003039 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579ea73-9964-44eb-b3b2-7e24ed57f615 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:38.007814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628bed2-78c6-4df3-9977-9040e8bed815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:38.008409 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bed2-78c6-4df3-9977-9040e8bed815 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:38.008691 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bed2-78c6-4df3-9977-9040e8bed815 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:38.008966 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bed2-78c6-4df3-9977-9040e8bed815 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:38.009732 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628bed2-78c6-4df3-9977-9040e8bed815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:38.011905 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628bed2-78c6-4df3-9977-9040e8bed815 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:38.022370 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1431/2842] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:38.027111 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a74ea9-0e8b-4b6f-9d2f-abca5f378612 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:38.037186 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a74ea9-0e8b-4b6f-9d2f-abca5f378612 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:38.042879 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e16e8-5194-4fd8-897b-641f6912217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:38.043842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e16e8-5194-4fd8-897b-641f6912217d admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:38.044178 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e16e8-5194-4fd8-897b-641f6912217d admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:38.051655 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e16e8-5194-4fd8-897b-641f6912217d admin admin] PATH_INFO: `/v3/projects/a63345953bec43bea51afe785fe6ae37/users/41833303df5141b2a6f2fc32df632937/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:38.060398 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e16e8-5194-4fd8-897b-641f6912217d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63345953bec43bea51afe785fe6ae37, user_id=41833303df5141b2a6f2fc32df632937, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:38.063098 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e16e8-5194-4fd8-897b-641f6912217d admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:38.077012 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1412/2843] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:03:38 2022] PUT /identity/v3/projects/a63345953bec43bea51afe785fe6ae37/users/41833303df5141b2a6f2fc32df632937/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:38.082145 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936bdc1-baa4-42e0-a4a7-338d0178d46c None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:38.082436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936bdc1-baa4-42e0-a4a7-338d0178d46c None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:38.082713 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936bdc1-baa4-42e0-a4a7-338d0178d46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:38.100560 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-4936bdc1-baa4-42e0-a4a7-338d0178d46c None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:38.101788 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-4936bdc1-baa4-42e0-a4a7-338d0178d46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:38.163272 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1432/2844] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:38.173623 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e16e8-5194-4fd8-897b-641f6912217d admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:38.175331 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-e21e16e8-5194-4fd8-897b-641f6912217d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:38.356518 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e16e8-5194-4fd8-897b-641f6912217d admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:38.360332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14c4c5-b9a1-4d2c-8fbc-fe1fc3659d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:38.361001 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14c4c5-b9a1-4d2c-8fbc-fe1fc3659d26 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:38.361283 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14c4c5-b9a1-4d2c-8fbc-fe1fc3659d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:38.361558 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14c4c5-b9a1-4d2c-8fbc-fe1fc3659d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:38.370727 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14c4c5-b9a1-4d2c-8fbc-fe1fc3659d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:38.372148 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14c4c5-b9a1-4d2c-8fbc-fe1fc3659d26 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:38.381489 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1413/2845] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:39.281431 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b020ed1-0e57-40e5-9d39-308a024f6d57 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:39.281916 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b020ed1-0e57-40e5-9d39-308a024f6d57 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:39.282031 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b020ed1-0e57-40e5-9d39-308a024f6d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:39.318654 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-5b020ed1-0e57-40e5-9d39-308a024f6d57 None None] MFA Rules not processed for user `41833303df5141b2a6f2fc32df632937`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:39.320098 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-5b020ed1-0e57-40e5-9d39-308a024f6d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:39.394493 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1433/2846] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:39 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:39.398875 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14c4c5-b9a1-4d2c-8fbc-fe1fc3659d26 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:39.408965 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14c4c5-b9a1-4d2c-8fbc-fe1fc3659d26 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:39.413551 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b9e1c-a253-484e-be80-20e8eb4c94ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:39.414842 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b9e1c-a253-484e-be80-20e8eb4c94ca admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:39.415120 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b9e1c-a253-484e-be80-20e8eb4c94ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:39.415586 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b9e1c-a253-484e-be80-20e8eb4c94ca admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:39.416914 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b9e1c-a253-484e-be80-20e8eb4c94ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:39.419461 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b9e1c-a253-484e-be80-20e8eb4c94ca admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:39.430092 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:39.451022 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1414/2847] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:03:39 2022] POST /identity/v3/projects => generated 401 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:03:39.454797 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b020ed1-0e57-40e5-9d39-308a024f6d57 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:39.466327 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b020ed1-0e57-40e5-9d39-308a024f6d57 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:39.499757 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:39.524816 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75e82a-8c54-4302-a861-edfcd4b8f685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:39.526026 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75e82a-8c54-4302-a861-edfcd4b8f685 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:39.526743 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75e82a-8c54-4302-a861-edfcd4b8f685 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:39.527025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75e82a-8c54-4302-a861-edfcd4b8f685 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:39.527707 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75e82a-8c54-4302-a861-edfcd4b8f685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:39.529632 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75e82a-8c54-4302-a861-edfcd4b8f685 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:39.549687 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1434/2848] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:03:39 2022] POST /identity/v3/users => generated 307 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:03:39.553716 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b9e1c-a253-484e-be80-20e8eb4c94ca admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:39.564486 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b9e1c-a253-484e-be80-20e8eb4c94ca admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:39.568919 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f85721-ef74-48e8-a41e-feea98146435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:39.569562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f85721-ef74-48e8-a41e-feea98146435 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:39.569864 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f85721-ef74-48e8-a41e-feea98146435 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:39.570130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f85721-ef74-48e8-a41e-feea98146435 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:39.570835 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f85721-ef74-48e8-a41e-feea98146435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:39.572446 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f85721-ef74-48e8-a41e-feea98146435 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:39.583020 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1415/2849] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:39.587195 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75e82a-8c54-4302-a861-edfcd4b8f685 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:39.597906 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75e82a-8c54-4302-a861-edfcd4b8f685 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:39.602488 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d50d2a-32cd-42e9-940b-4da476ff3f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:39.603373 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d50d2a-32cd-42e9-940b-4da476ff3f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:39.603653 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d50d2a-32cd-42e9-940b-4da476ff3f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:39.604552 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d50d2a-32cd-42e9-940b-4da476ff3f69 admin admin] PATH_INFO: `/v3/projects/73d4df984fd64d3cbfc5960822cffec6/users/09f5cb7db7a8431eb9d63b22a00b8a64/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:39.619222 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d50d2a-32cd-42e9-940b-4da476ff3f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d4df984fd64d3cbfc5960822cffec6, user_id=09f5cb7db7a8431eb9d63b22a00b8a64, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:39.622135 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d50d2a-32cd-42e9-940b-4da476ff3f69 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:39.635061 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1435/2850] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:03:39 2022] PUT /identity/v3/projects/73d4df984fd64d3cbfc5960822cffec6/users/09f5cb7db7a8431eb9d63b22a00b8a64/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:39.639027 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f85721-ef74-48e8-a41e-feea98146435 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:39.649124 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f85721-ef74-48e8-a41e-feea98146435 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:39.705118 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c768cbd-c114-4501-90df-12d36aec1cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:39.705801 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768cbd-c114-4501-90df-12d36aec1cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:39.706097 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768cbd-c114-4501-90df-12d36aec1cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:39.706351 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768cbd-c114-4501-90df-12d36aec1cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:39.707063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c768cbd-c114-4501-90df-12d36aec1cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:39.708640 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c768cbd-c114-4501-90df-12d36aec1cd9 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:39.718682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1416/2851] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 20:03:39.722777 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d50d2a-32cd-42e9-940b-4da476ff3f69 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:39.733319 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d50d2a-32cd-42e9-940b-4da476ff3f69 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:39.737992 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d3d26-fe11-4e44-a204-f90dca7abe1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:39.738879 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d3d26-fe11-4e44-a204-f90dca7abe1a admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:39.739161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d3d26-fe11-4e44-a204-f90dca7abe1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:39.739437 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d3d26-fe11-4e44-a204-f90dca7abe1a admin admin] PATH_INFO: `/v3/projects/73d4df984fd64d3cbfc5960822cffec6/users/09f5cb7db7a8431eb9d63b22a00b8a64/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:39.741391 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d3d26-fe11-4e44-a204-f90dca7abe1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d4df984fd64d3cbfc5960822cffec6, user_id=09f5cb7db7a8431eb9d63b22a00b8a64, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:39.744076 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d3d26-fe11-4e44-a204-f90dca7abe1a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:39.754888 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1436/2852] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:03:39 2022] PUT /identity/v3/projects/73d4df984fd64d3cbfc5960822cffec6/users/09f5cb7db7a8431eb9d63b22a00b8a64/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:40.354119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a10a-0090-428b-969c-73d3442b12a4 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:40.354414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a10a-0090-428b-969c-73d3442b12a4 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:40.354686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a10a-0090-428b-969c-73d3442b12a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:40.383249 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-7950a10a-0090-428b-969c-73d3442b12a4 None None] MFA Rules not processed for user `09f5cb7db7a8431eb9d63b22a00b8a64`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:40.384473 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-7950a10a-0090-428b-969c-73d3442b12a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:40.456869 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1417/2853] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:40 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:41.443963 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d3d26-fe11-4e44-a204-f90dca7abe1a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:41.445755 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-a08d3d26-fe11-4e44-a204-f90dca7abe1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:41.496956 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950a10a-0090-428b-969c-73d3442b12a4 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:41.529450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d3d26-fe11-4e44-a204-f90dca7abe1a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:41.534523 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39207dcd-f491-4a05-ad8f-5116483ce607 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:41.535169 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207dcd-f491-4a05-ad8f-5116483ce607 service placement] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:41.535451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207dcd-f491-4a05-ad8f-5116483ce607 service placement] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:41.535726 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207dcd-f491-4a05-ad8f-5116483ce607 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:41.537536 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-39207dcd-f491-4a05-ad8f-5116483ce607 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:41.549775 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39207dcd-f491-4a05-ad8f-5116483ce607 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:41.551384 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39207dcd-f491-4a05-ad8f-5116483ce607 service placement] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:41.562495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1437/2854] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 20:03:41.565561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950a10a-0090-428b-969c-73d3442b12a4 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:41.570087 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:41.570960 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:41.571236 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:41.571507 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] PATH_INFO: `/v3/users/41833303df5141b2a6f2fc32df632937` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:41.572732 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41833303df5141b2a6f2fc32df632937)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:41.574866 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:41.577298 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:41.625949 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41833303df5141b2a6f2fc32df632937'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:41.637786 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41833303df5141b2a6f2fc32df632937'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:41.646032 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41833303df5141b2a6f2fc32df632937'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:41.653605 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41833303df5141b2a6f2fc32df632937'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:41.654263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41833303df5141b2a6f2fc32df632937'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:41.660193 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1418/2855] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:03:41 2022] DELETE /identity/v3/users/41833303df5141b2a6f2fc32df632937 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:41.831957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39207dcd-f491-4a05-ad8f-5116483ce607 service placement] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:41.843335 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39207dcd-f491-4a05-ad8f-5116483ce607 service placement] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:41.903232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:41.904087 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:41.904368 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:41.904640 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] PATH_INFO: `/v3/projects/a63345953bec43bea51afe785fe6ae37` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:41.905821 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63345953bec43bea51afe785fe6ae37)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:41.907711 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:41.929440 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:41.956648 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63345953bec43bea51afe785fe6ae37', 'request_id': 'req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:41.965369 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63345953bec43bea51afe785fe6ae37', 'request_id': 'req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:41.971589 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63345953bec43bea51afe785fe6ae37', 'request_id': 'req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:41.982053 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] The token cache is being invalidate because project a63345953bec43bea51afe785fe6ae37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:03:41.982343 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:41.982979 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:41.984512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1438/2856] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:03:41 2022] DELETE /identity/v3/projects/a63345953bec43bea51afe785fe6ae37 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:41.988627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:41.990046 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:42.002962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878da91-8cea-43a9-a981-dbc4c8885b60 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:42.058267 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:42.059130 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.059410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.059681 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] PATH_INFO: `/v3/users/09f5cb7db7a8431eb9d63b22a00b8a64` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.060848 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f5cb7db7a8431eb9d63b22a00b8a64)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:42.062771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:42.107042 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f5cb7db7a8431eb9d63b22a00b8a64'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:42.118771 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f5cb7db7a8431eb9d63b22a00b8a64'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:42.126214 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f5cb7db7a8431eb9d63b22a00b8a64'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:42.132988 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f5cb7db7a8431eb9d63b22a00b8a64'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:42.133642 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-fa032d4e-d1de-4af4-9d25-a407499123d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f5cb7db7a8431eb9d63b22a00b8a64'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:42.136329 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:42.141356 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1419/2857] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:03:41 2022] DELETE /identity/v3/users/09f5cb7db7a8431eb9d63b22a00b8a64 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:42.311018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:42.313011 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:42.328988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ec1ee-eadb-4b9f-9b72-b2d03f284e95 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:42.389478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:42.390614 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.390947 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.391274 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] PATH_INFO: `/v3/projects/73d4df984fd64d3cbfc5960822cffec6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.392456 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73d4df984fd64d3cbfc5960822cffec6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:42.394725 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:42.443315 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73d4df984fd64d3cbfc5960822cffec6', 'request_id': 'req-621a4e17-634c-422f-99e0-81d1aeb1c398'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:42.450794 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73d4df984fd64d3cbfc5960822cffec6', 'request_id': 'req-621a4e17-634c-422f-99e0-81d1aeb1c398'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:42.457651 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73d4df984fd64d3cbfc5960822cffec6', 'request_id': 'req-621a4e17-634c-422f-99e0-81d1aeb1c398'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:42.468902 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] The token cache is being invalidate because project 73d4df984fd64d3cbfc5960822cffec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:03:42.469197 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:42.469895 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-621a4e17-634c-422f-99e0-81d1aeb1c398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:42.471461 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1439/2858] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:03:42 2022] DELETE /identity/v3/projects/73d4df984fd64d3cbfc5960822cffec6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:42.477312 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8fb67-2e98-4555-9cc9-e6fe658216ad None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.477627 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8fb67-2e98-4555-9cc9-e6fe658216ad None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.477910 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8fb67-2e98-4555-9cc9-e6fe658216ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.496645 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-81d8fb67-2e98-4555-9cc9-e6fe658216ad None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:42.497879 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-81d8fb67-2e98-4555-9cc9-e6fe658216ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:42.560146 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1420/2859] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:42.619248 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6a4c4-3b01-4e46-9b8d-007b7b8c04fa None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.619731 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6a4c4-3b01-4e46-9b8d-007b7b8c04fa None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.620336 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6a4c4-3b01-4e46-9b8d-007b7b8c04fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.638240 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-dab6a4c4-3b01-4e46-9b8d-007b7b8c04fa None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:42.639451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-dab6a4c4-3b01-4e46-9b8d-007b7b8c04fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:42.647953 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1440/2860] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:42.652660 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8fb67-2e98-4555-9cc9-e6fe658216ad None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:42.662173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8fb67-2e98-4555-9cc9-e6fe658216ad None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:42.667272 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3c1d6-8423-4886-a377-4a4ce6b9839b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:42.667909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3c1d6-8423-4886-a377-4a4ce6b9839b admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.668189 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3c1d6-8423-4886-a377-4a4ce6b9839b admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.668463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3c1d6-8423-4886-a377-4a4ce6b9839b admin admin] PATH_INFO: `/v3/domains` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.669179 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3c1d6-8423-4886-a377-4a4ce6b9839b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86779) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:03:42.669487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3c1d6-8423-4886-a377-4a4ce6b9839b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:42.671102 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3c1d6-8423-4886-a377-4a4ce6b9839b admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:42.682629 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1421/2861] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 20:03:42.686512 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6a4c4-3b01-4e46-9b8d-007b7b8c04fa None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:42.695510 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6a4c4-3b01-4e46-9b8d-007b7b8c04fa None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:42.700901 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eeb66-783d-41bd-8603-ef7467c94d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:42.701537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eeb66-783d-41bd-8603-ef7467c94d49 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.701846 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eeb66-783d-41bd-8603-ef7467c94d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.702119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eeb66-783d-41bd-8603-ef7467c94d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.702827 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eeb66-783d-41bd-8603-ef7467c94d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:42.704705 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eeb66-783d-41bd-8603-ef7467c94d49 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:42.729988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1441/2862] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:03:42 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:03:42.734134 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3c1d6-8423-4886-a377-4a4ce6b9839b admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:42.744116 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3c1d6-8423-4886-a377-4a4ce6b9839b admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:42.801450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fabd5-fd4c-4060-a46b-03be1d6980c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:42.802135 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fabd5-fd4c-4060-a46b-03be1d6980c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.802415 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fabd5-fd4c-4060-a46b-03be1d6980c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.802686 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fabd5-fd4c-4060-a46b-03be1d6980c2 admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.803374 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fabd5-fd4c-4060-a46b-03be1d6980c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:42.805262 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fabd5-fd4c-4060-a46b-03be1d6980c2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:42.826463 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1422/2863] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:03:42 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:03:42.830537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eeb66-783d-41bd-8603-ef7467c94d49 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:42.841459 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eeb66-783d-41bd-8603-ef7467c94d49 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:42.846097 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6a56f-0a0e-4197-88b5-cf4c38dba714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:42.846741 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6a56f-0a0e-4197-88b5-cf4c38dba714 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.847018 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6a56f-0a0e-4197-88b5-cf4c38dba714 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.847282 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6a56f-0a0e-4197-88b5-cf4c38dba714 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.847993 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6a56f-0a0e-4197-88b5-cf4c38dba714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:42.849566 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6a56f-0a0e-4197-88b5-cf4c38dba714 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:42.851584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:42.862765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1442/2864] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:42.866192 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fabd5-fd4c-4060-a46b-03be1d6980c2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:42.876957 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fabd5-fd4c-4060-a46b-03be1d6980c2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:42.881812 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e859d0c-e0f3-4786-9fd5-42b520255269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:42.882465 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e859d0c-e0f3-4786-9fd5-42b520255269 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.882747 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e859d0c-e0f3-4786-9fd5-42b520255269 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.883019 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e859d0c-e0f3-4786-9fd5-42b520255269 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.883738 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e859d0c-e0f3-4786-9fd5-42b520255269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:42.885353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e859d0c-e0f3-4786-9fd5-42b520255269 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:42.895392 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1423/2865] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:42.899399 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6a56f-0a0e-4197-88b5-cf4c38dba714 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:42.909251 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6a56f-0a0e-4197-88b5-cf4c38dba714 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:42.913833 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1d4f3-84d9-451f-bb6f-d798df856056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:42.914709 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1d4f3-84d9-451f-bb6f-d798df856056 admin admin] REQUEST_METHOD: `PUT` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.914980 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1d4f3-84d9-451f-bb6f-d798df856056 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.915247 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1d4f3-84d9-451f-bb6f-d798df856056 admin admin] PATH_INFO: `/v3/projects/c083694e42464c73a0a1ce62a6c507bb/users/107161bd4afe43b29fe0b720e05a4bde/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.929866 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1d4f3-84d9-451f-bb6f-d798df856056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c083694e42464c73a0a1ce62a6c507bb, user_id=107161bd4afe43b29fe0b720e05a4bde, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:42.933167 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1d4f3-84d9-451f-bb6f-d798df856056 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:42.945633 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1443/2866] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:03:42 2022] PUT /identity/v3/projects/c083694e42464c73a0a1ce62a6c507bb/users/107161bd4afe43b29fe0b720e05a4bde/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 20:03:42.955641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f94f0-e752-4cb9-a873-caefb6ec793f None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:42.955952 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f94f0-e752-4cb9-a873-caefb6ec793f None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:42.956251 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f94f0-e752-4cb9-a873-caefb6ec793f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:42.982104 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-591f94f0-e752-4cb9-a873-caefb6ec793f None None] MFA Rules not processed for user `107161bd4afe43b29fe0b720e05a4bde`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:42.983305 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-591f94f0-e752-4cb9-a873-caefb6ec793f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:43.038274 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:43.048683 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1424/2867] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:43.053320 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1d4f3-84d9-451f-bb6f-d798df856056 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:43.115457 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1d4f3-84d9-451f-bb6f-d798df856056 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:43.120196 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c746b2f-d013-4853-a17e-f54508516342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:43.120845 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c746b2f-d013-4853-a17e-f54508516342 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:43.121129 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c746b2f-d013-4853-a17e-f54508516342 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:43.121396 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c746b2f-d013-4853-a17e-f54508516342 admin admin] PATH_INFO: `/v3/projects` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:43.122110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c746b2f-d013-4853-a17e-f54508516342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:43.124004 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c746b2f-d013-4853-a17e-f54508516342 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:43.148596 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1444/2868] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:03:43 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:03:43.153094 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f94f0-e752-4cb9-a873-caefb6ec793f None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:43.163630 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f94f0-e752-4cb9-a873-caefb6ec793f None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:43.220414 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83008fa-2ecd-4b46-87a6-a93bdd4ea3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:43.221076 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83008fa-2ecd-4b46-87a6-a93bdd4ea3be admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:43.221359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83008fa-2ecd-4b46-87a6-a93bdd4ea3be admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:43.221664 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83008fa-2ecd-4b46-87a6-a93bdd4ea3be admin admin] PATH_INFO: `/v3/users` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:43.222364 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83008fa-2ecd-4b46-87a6-a93bdd4ea3be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:43.224273 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83008fa-2ecd-4b46-87a6-a93bdd4ea3be admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:43.244410 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1425/2869] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:03:43 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:03:43.248403 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c746b2f-d013-4853-a17e-f54508516342 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:43.258754 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c746b2f-d013-4853-a17e-f54508516342 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:43.263835 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33da4e-0a72-4cc6-b3d4-fe08d7e1ce3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:43.264490 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33da4e-0a72-4cc6-b3d4-fe08d7e1ce3c admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:43.264772 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33da4e-0a72-4cc6-b3d4-fe08d7e1ce3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:43.265055 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33da4e-0a72-4cc6-b3d4-fe08d7e1ce3c admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:43.265814 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33da4e-0a72-4cc6-b3d4-fe08d7e1ce3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:43.267415 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33da4e-0a72-4cc6-b3d4-fe08d7e1ce3c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:43.278964 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1445/2870] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:43.282909 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83008fa-2ecd-4b46-87a6-a93bdd4ea3be admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:43.293665 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83008fa-2ecd-4b46-87a6-a93bdd4ea3be admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:43.298244 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad81dfa-f6f6-4caf-ada1-3ac9904471de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:43.299114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81dfa-f6f6-4caf-ada1-3ac9904471de admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:43.299395 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81dfa-f6f6-4caf-ada1-3ac9904471de admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:43.299668 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad81dfa-f6f6-4caf-ada1-3ac9904471de admin admin] PATH_INFO: `/v3/projects/69ca2cdce7bf4f13ab10473f50dc789b/users/7d48b79b2eb44df18ed6d979fa496af9/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:43.315079 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad81dfa-f6f6-4caf-ada1-3ac9904471de admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ca2cdce7bf4f13ab10473f50dc789b, user_id=7d48b79b2eb44df18ed6d979fa496af9, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:43.317832 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad81dfa-f6f6-4caf-ada1-3ac9904471de admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:43.332175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1426/2871] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:03:43 2022] PUT /identity/v3/projects/69ca2cdce7bf4f13ab10473f50dc789b/users/7d48b79b2eb44df18ed6d979fa496af9/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:43.336026 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33da4e-0a72-4cc6-b3d4-fe08d7e1ce3c admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:43.346024 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33da4e-0a72-4cc6-b3d4-fe08d7e1ce3c admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:43.403290 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de410050-f2d0-47c9-a54b-a6761ec2140c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:43.403894 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de410050-f2d0-47c9-a54b-a6761ec2140c admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:43.404175 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de410050-f2d0-47c9-a54b-a6761ec2140c admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:43.404450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de410050-f2d0-47c9-a54b-a6761ec2140c admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:43.405177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de410050-f2d0-47c9-a54b-a6761ec2140c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:43.406793 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de410050-f2d0-47c9-a54b-a6761ec2140c admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:43.420487 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1446/2872] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:43.422766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad81dfa-f6f6-4caf-ada1-3ac9904471de admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:43.433635 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad81dfa-f6f6-4caf-ada1-3ac9904471de admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:43.438370 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b187c-d45c-4201-8f74-dfedd10420e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:43.439252 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b187c-d45c-4201-8f74-dfedd10420e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:43.439527 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b187c-d45c-4201-8f74-dfedd10420e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:43.439800 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b187c-d45c-4201-8f74-dfedd10420e2 admin admin] PATH_INFO: `/v3/projects/69ca2cdce7bf4f13ab10473f50dc789b/users/7d48b79b2eb44df18ed6d979fa496af9/roles/f7da1306127e4efa845a2644d7be96d5` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:43.441781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b187c-d45c-4201-8f74-dfedd10420e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ca2cdce7bf4f13ab10473f50dc789b, user_id=7d48b79b2eb44df18ed6d979fa496af9, role_id=f7da1306127e4efa845a2644d7be96d5)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:43.444390 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b187c-d45c-4201-8f74-dfedd10420e2 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:43.454731 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1427/2873] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:03:43 2022] PUT /identity/v3/projects/69ca2cdce7bf4f13ab10473f50dc789b/users/7d48b79b2eb44df18ed6d979fa496af9/roles/f7da1306127e4efa845a2644d7be96d5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:43.465002 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f2538-39f2-467f-aa23-0bab03d8bb19 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:43.465623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f2538-39f2-467f-aa23-0bab03d8bb19 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:43.465623 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f2538-39f2-467f-aa23-0bab03d8bb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:43.497131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-3b9f2538-39f2-467f-aa23-0bab03d8bb19 None None] MFA Rules not processed for user `7d48b79b2eb44df18ed6d979fa496af9`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:43.498401 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-3b9f2538-39f2-467f-aa23-0bab03d8bb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:43.533830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:43.584715 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1447/2874] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:43.595238 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b187c-d45c-4201-8f74-dfedd10420e2 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:43.597050 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-9f1b187c-d45c-4201-8f74-dfedd10420e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:43.656450 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b187c-d45c-4201-8f74-dfedd10420e2 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:43.660172 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adfb58-aa0b-4184-b339-75d7d164ad6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:43.660846 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adfb58-aa0b-4184-b339-75d7d164ad6e service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:43.661125 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adfb58-aa0b-4184-b339-75d7d164ad6e service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:43.661400 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adfb58-aa0b-4184-b339-75d7d164ad6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:43.671131 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adfb58-aa0b-4184-b339-75d7d164ad6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:43.672539 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6adfb58-aa0b-4184-b339-75d7d164ad6e service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:43.681906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1428/2875] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:44.033133 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f2538-39f2-467f-aa23-0bab03d8bb19 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:44.095771 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f2538-39f2-467f-aa23-0bab03d8bb19 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:44.100428 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:44.101301 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:44.101603 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:44.101957 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] PATH_INFO: `/v3/users/107161bd4afe43b29fe0b720e05a4bde` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:44.103146 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=107161bd4afe43b29fe0b720e05a4bde)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:44.105244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:44.150673 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '107161bd4afe43b29fe0b720e05a4bde'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.157718 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '107161bd4afe43b29fe0b720e05a4bde'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.158338 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '107161bd4afe43b29fe0b720e05a4bde'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.170683 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '107161bd4afe43b29fe0b720e05a4bde'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.171254 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '107161bd4afe43b29fe0b720e05a4bde'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.184211 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1448/2876] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:03:44 2022] DELETE /identity/v3/users/107161bd4afe43b29fe0b720e05a4bde => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:44.188989 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804b5ef-b20b-4dc2-bd6e-d3ab931d8bb3 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:44.189284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804b5ef-b20b-4dc2-bd6e-d3ab931d8bb3 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:44.189561 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804b5ef-b20b-4dc2-bd6e-d3ab931d8bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:44.206908 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-c804b5ef-b20b-4dc2-bd6e-d3ab931d8bb3 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:44.208090 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c804b5ef-b20b-4dc2-bd6e-d3ab931d8bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:44.252393 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:44.268010 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1429/2877] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:44.278584 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:44.279980 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:44.333229 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:44.337423 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651bd0c-ae6f-4850-a366-c904957dc205 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:44.343353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52d54d-d261-4138-bcb6-fe53b04bbfca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:44.344228 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52d54d-d261-4138-bcb6-fe53b04bbfca service neutron] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:44.344680 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52d54d-d261-4138-bcb6-fe53b04bbfca service neutron] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:44.345046 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52d54d-d261-4138-bcb6-fe53b04bbfca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:44.354436 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52d54d-d261-4138-bcb6-fe53b04bbfca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:44.355874 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52d54d-d261-4138-bcb6-fe53b04bbfca service neutron] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:44.365340 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1449/2878] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:44.634531 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804b5ef-b20b-4dc2-bd6e-d3ab931d8bb3 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:44.636066 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-c804b5ef-b20b-4dc2-bd6e-d3ab931d8bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:44.650085 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804b5ef-b20b-4dc2-bd6e-d3ab931d8bb3 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:44.655040 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:44.655903 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:44.656185 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:44.656462 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] PATH_INFO: `/v3/projects/c083694e42464c73a0a1ce62a6c507bb` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:44.657781 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c083694e42464c73a0a1ce62a6c507bb)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:44.659733 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:44.708103 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c083694e42464c73a0a1ce62a6c507bb', 'request_id': 'req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.715656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c083694e42464c73a0a1ce62a6c507bb', 'request_id': 'req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.721106 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c083694e42464c73a0a1ce62a6c507bb', 'request_id': 'req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.732452 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] The token cache is being invalidate because project c083694e42464c73a0a1ce62a6c507bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:03:44.732734 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.733353 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.734885 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1430/2879] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:03:44 2022] DELETE /identity/v3/projects/c083694e42464c73a0a1ce62a6c507bb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:44.738857 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52d54d-d261-4138-bcb6-fe53b04bbfca service neutron] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:44.749284 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52d54d-d261-4138-bcb6-fe53b04bbfca service neutron] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:44.805071 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:44.805977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:44.806266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:44.806537 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] PATH_INFO: `/v3/users/7d48b79b2eb44df18ed6d979fa496af9` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:44.807717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d48b79b2eb44df18ed6d979fa496af9)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:44.810177 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:44.854389 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d48b79b2eb44df18ed6d979fa496af9'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.861316 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d48b79b2eb44df18ed6d979fa496af9'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.861952 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d48b79b2eb44df18ed6d979fa496af9'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.874478 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d48b79b2eb44df18ed6d979fa496af9'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.875118 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-cad97b79-4d9f-4887-9b6d-ce47bd2bddf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d48b79b2eb44df18ed6d979fa496af9'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:44.888094 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1450/2880] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:03:44 2022] DELETE /identity/v3/users/7d48b79b2eb44df18ed6d979fa496af9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:03:45.174804 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:45.176316 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:45.191562 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd6bcc-86a7-4c18-b9b6-7f3c32202205 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:45.252082 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:45.252865 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.253164 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.253434 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] PATH_INFO: `/v3/projects/69ca2cdce7bf4f13ab10473f50dc789b` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.254701 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ca2cdce7bf4f13ab10473f50dc789b)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:45.256596 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:45.304031 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ca2cdce7bf4f13ab10473f50dc789b', 'request_id': 'req-261266f4-eb5f-4cba-aeed-32ccaaa4e747'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:45.311075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ca2cdce7bf4f13ab10473f50dc789b', 'request_id': 'req-261266f4-eb5f-4cba-aeed-32ccaaa4e747'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:45.317509 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ca2cdce7bf4f13ab10473f50dc789b', 'request_id': 'req-261266f4-eb5f-4cba-aeed-32ccaaa4e747'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:45.329075 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] The token cache is being invalidate because project 69ca2cdce7bf4f13ab10473f50dc789b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86779) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:03:45.329363 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:45.330039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-261266f4-eb5f-4cba-aeed-32ccaaa4e747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:03:45.331610 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1431/2881] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:03:45 2022] DELETE /identity/v3/projects/69ca2cdce7bf4f13ab10473f50dc789b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 20:03:45.336407 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ec87d-5a5b-4d44-a4cd-8908fe79f512 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.336699 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ec87d-5a5b-4d44-a4cd-8908fe79f512 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.336977 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ec87d-5a5b-4d44-a4cd-8908fe79f512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.355955 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-7f4ec87d-5a5b-4d44-a4cd-8908fe79f512 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:45.357171 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-7f4ec87d-5a5b-4d44-a4cd-8908fe79f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:45.414830 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1451/2882] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:45.423761 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027657de-5c09-466f-9902-ec5ef6f3e978 None None] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.424039 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027657de-5c09-466f-9902-ec5ef6f3e978 None None] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.424309 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027657de-5c09-466f-9902-ec5ef6f3e978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.431413 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:45.443332 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.auth.core [None req-027657de-5c09-466f-9902-ec5ef6f3e978 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86779) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:45.444682 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-027657de-5c09-466f-9902-ec5ef6f3e978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:45.452962 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1432/2883] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:03:45.457261 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ec87d-5a5b-4d44-a4cd-8908fe79f512 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:45.467832 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ec87d-5a5b-4d44-a4cd-8908fe79f512 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:45.472460 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15b0d3-80bd-41e7-8acc-7769265d057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:45.473119 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15b0d3-80bd-41e7-8acc-7769265d057a admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.473400 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15b0d3-80bd-41e7-8acc-7769265d057a admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.473717 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15b0d3-80bd-41e7-8acc-7769265d057a admin admin] PATH_INFO: `/v3/domains` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.474451 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15b0d3-80bd-41e7-8acc-7769265d057a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86778) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 20:03:45.474752 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15b0d3-80bd-41e7-8acc-7769265d057a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:45.476353 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15b0d3-80bd-41e7-8acc-7769265d057a admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:45.483899 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:45.493221 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1452/2884] 10.222.0.66 () {58 vars in 1152 bytes} [Fri Jul 1 20:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 20:03:45.494517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027657de-5c09-466f-9902-ec5ef6f3e978 None None] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:45.503933 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027657de-5c09-466f-9902-ec5ef6f3e978 None None] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:45.508611 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867a61b-6243-43f6-8c43-36667c4cda9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:45.509264 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867a61b-6243-43f6-8c43-36667c4cda9c admin admin] REQUEST_METHOD: `POST` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.509569 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867a61b-6243-43f6-8c43-36667c4cda9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.509870 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867a61b-6243-43f6-8c43-36667c4cda9c admin admin] PATH_INFO: `/v3/projects` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.510613 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867a61b-6243-43f6-8c43-36667c4cda9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:45.512620 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867a61b-6243-43f6-8c43-36667c4cda9c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:45.538360 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1433/2885] 10.222.0.66 () {60 vars in 1152 bytes} [Fri Jul 1 20:03:45 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 20:03:45.542758 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15b0d3-80bd-41e7-8acc-7769265d057a admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:45.553663 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15b0d3-80bd-41e7-8acc-7769265d057a admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:45.610159 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933085-2421-4d54-8638-5f0aa2a8b196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:45.610849 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933085-2421-4d54-8638-5f0aa2a8b196 admin admin] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.611115 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933085-2421-4d54-8638-5f0aa2a8b196 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.611392 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61933085-2421-4d54-8638-5f0aa2a8b196 admin admin] PATH_INFO: `/v3/users` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.612072 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933085-2421-4d54-8638-5f0aa2a8b196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:45.614023 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61933085-2421-4d54-8638-5f0aa2a8b196 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:45.633591 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1453/2886] 10.222.0.66 () {60 vars in 1143 bytes} [Fri Jul 1 20:03:45 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 20:03:45.638940 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867a61b-6243-43f6-8c43-36667c4cda9c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:45.650497 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867a61b-6243-43f6-8c43-36667c4cda9c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:45.655954 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af39ff-51df-4188-8f2c-ad691f7ad518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:45.656612 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af39ff-51df-4188-8f2c-ad691f7ad518 admin admin] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.656891 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af39ff-51df-4188-8f2c-ad691f7ad518 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.657285 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af39ff-51df-4188-8f2c-ad691f7ad518 admin admin] PATH_INFO: `/v3/roles` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.657906 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af39ff-51df-4188-8f2c-ad691f7ad518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:45.659483 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af39ff-51df-4188-8f2c-ad691f7ad518 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:45.670263 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1434/2887] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:45.674238 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933085-2421-4d54-8638-5f0aa2a8b196 admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:45.684695 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61933085-2421-4d54-8638-5f0aa2a8b196 admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:45.689312 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c792d-f0f0-4306-8970-c62ed00bde37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:45.690021 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c792d-f0f0-4306-8970-c62ed00bde37 admin admin] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.690300 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c792d-f0f0-4306-8970-c62ed00bde37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.690571 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c792d-f0f0-4306-8970-c62ed00bde37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.691288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c792d-f0f0-4306-8970-c62ed00bde37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:45.692872 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c792d-f0f0-4306-8970-c62ed00bde37 admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:45.703374 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1454/2888] 10.222.0.66 () {58 vars in 1121 bytes} [Fri Jul 1 20:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 20:03:45.707641 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af39ff-51df-4188-8f2c-ad691f7ad518 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:45.717980 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af39ff-51df-4188-8f2c-ad691f7ad518 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:45.722563 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771abaae-fb89-42cd-a732-37d0c1d8c5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:45.723412 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771abaae-fb89-42cd-a732-37d0c1d8c5ce admin admin] REQUEST_METHOD: `PUT` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.723690 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771abaae-fb89-42cd-a732-37d0c1d8c5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.723961 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771abaae-fb89-42cd-a732-37d0c1d8c5ce admin admin] PATH_INFO: `/v3/projects/a02ae0275bcd429cb4a015a5d68e277e/users/39f78d7a2f384ed780fdee865c5f133c/roles/2e1db0cf27684f389b9320673721684d` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.738157 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771abaae-fb89-42cd-a732-37d0c1d8c5ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02ae0275bcd429cb4a015a5d68e277e, user_id=39f78d7a2f384ed780fdee865c5f133c, role_id=2e1db0cf27684f389b9320673721684d)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:45.740766 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771abaae-fb89-42cd-a732-37d0c1d8c5ce admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:45.753037 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1435/2889] 10.222.0.66 () {60 vars in 1482 bytes} [Fri Jul 1 20:03:45 2022] PUT /identity/v3/projects/a02ae0275bcd429cb4a015a5d68e277e/users/39f78d7a2f384ed780fdee865c5f133c/roles/2e1db0cf27684f389b9320673721684d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 20:03:45.763775 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf943d9-2b95-4ca4-9e06-b5993fad2baf None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.764264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf943d9-2b95-4ca4-9e06-b5993fad2baf None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.764264 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf943d9-2b95-4ca4-9e06-b5993fad2baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.790998 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-0cf943d9-2b95-4ca4-9e06-b5993fad2baf None None] MFA Rules not processed for user `39f78d7a2f384ed780fdee865c5f133c`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:03:45.792210 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0cf943d9-2b95-4ca4-9e06-b5993fad2baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:45.855007 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1455/2890] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:03:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 20:03:45.866139 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771abaae-fb89-42cd-a732-37d0c1d8c5ce admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:03:45.867799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-771abaae-fb89-42cd-a732-37d0c1d8c5ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:03:45.924505 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771abaae-fb89-42cd-a732-37d0c1d8c5ce admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:03:45.928174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb03ae-d480-4e6f-a3a5-2e517fe3e546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4b0e6ea57af454cbc8f5bd180bc052e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:03:45.928830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb03ae-d480-4e6f-a3a5-2e517fe3e546 service cinder] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:03:45.929114 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb03ae-d480-4e6f-a3a5-2e517fe3e546 service cinder] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:03:45.929387 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb03ae-d480-4e6f-a3a5-2e517fe3e546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:03:45.938514 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb03ae-d480-4e6f-a3a5-2e517fe3e546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:03:45.939935 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb03ae-d480-4e6f-a3a5-2e517fe3e546 service cinder] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:03:45.948676 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1436/2891] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:03:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:03:55.852124 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:03:55.944656 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:04:00.971645 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf943d9-2b95-4ca4-9e06-b5993fad2baf None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:04:00.973789 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-0cf943d9-2b95-4ca4-9e06-b5993fad2baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:04:01.065470 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf943d9-2b95-4ca4-9e06-b5993fad2baf None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:04:01.071110 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494e03f-65a4-4eba-ba30-12feb3746baf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad06f31859f9413f9db6039ac2abf982', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:04:01.071765 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494e03f-65a4-4eba-ba30-12feb3746baf service nova] REQUEST_METHOD: `GET` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:04:01.072048 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494e03f-65a4-4eba-ba30-12feb3746baf service nova] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:04:01.072326 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494e03f-65a4-4eba-ba30-12feb3746baf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:04:01.074232 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-d494e03f-65a4-4eba-ba30-12feb3746baf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:04:01.087836 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494e03f-65a4-4eba-ba30-12feb3746baf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:04:01.089450 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494e03f-65a4-4eba-ba30-12feb3746baf service nova] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:04:01.100834 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1456/2892] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:04:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 20:04:07.384666 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb03ae-d480-4e6f-a3a5-2e517fe3e546 service cinder] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:04:07.450063 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb03ae-d480-4e6f-a3a5-2e517fe3e546 service cinder] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:04:07.454638 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:04:07.455517 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:04:07.455799 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:04:07.456096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] PATH_INFO: `/v3/users/39f78d7a2f384ed780fdee865c5f133c` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:04:07.457292 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] RBAC: Authorizing `identity:delete_user(user_id=39f78d7a2f384ed780fdee865c5f133c)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:04:07.459247 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:04:07.506830 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39f78d7a2f384ed780fdee865c5f133c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:07.519174 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39f78d7a2f384ed780fdee865c5f133c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:07.526133 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39f78d7a2f384ed780fdee865c5f133c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:07.533119 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39f78d7a2f384ed780fdee865c5f133c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:07.533774 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39f78d7a2f384ed780fdee865c5f133c'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:07.539628 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1437/2893] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:04:07 2022] DELETE /identity/v3/users/39f78d7a2f384ed780fdee865c5f133c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:04:07.544760 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3eaf7-e045-409c-b953-643e84300266 None None] REQUEST_METHOD: `POST` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:04:07.545244 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3eaf7-e045-409c-b953-643e84300266 None None] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:04:07.545350 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3eaf7-e045-409c-b953-643e84300266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:04:07.563406 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.auth.core [None req-6dd3eaf7-e045-409c-b953-643e84300266 None None] MFA Rules not processed for user `9d09bdd9927e44ffbfd9b2ec447a5887`. Rule list: `[]` (Enabled: `True`). {{(pid=86778) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 20:04:07.564602 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6dd3eaf7-e045-409c-b953-643e84300266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:04:07.622062 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1457/2894] 10.222.0.66 () {58 vars in 957 bytes} [Fri Jul 1 20:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 20:04:07.631950 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:04:07.633344 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:04:07.694049 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0fdca-dade-4b42-a091-3b8247c9027c admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:04:07.697722 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4306bf-5705-4acf-b255-b6f60ef65983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '370f1c0f7af04a56bdf8fa452550380a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:04:07.698359 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4306bf-5705-4acf-b255-b6f60ef65983 service neutron] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:04:07.698684 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4306bf-5705-4acf-b255-b6f60ef65983 service neutron] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:04:07.698958 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4306bf-5705-4acf-b255-b6f60ef65983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:04:07.710084 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4306bf-5705-4acf-b255-b6f60ef65983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:04:07.711511 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4306bf-5705-4acf-b255-b6f60ef65983 service neutron] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:04:07.721487 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1438/2895] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:04:07.992562 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3eaf7-e045-409c-b953-643e84300266 None None] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:04:07.994091 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-6dd3eaf7-e045-409c-b953-643e84300266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:04:08.006820 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3eaf7-e045-409c-b953-643e84300266 None None] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:04:08.011710 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:04:08.012586 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:04:08.012865 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:04:08.013266 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] PATH_INFO: `/v3/projects/a02ae0275bcd429cb4a015a5d68e277e` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:04:08.014495 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a02ae0275bcd429cb4a015a5d68e277e)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:04:08.016424 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:04:08.063382 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a02ae0275bcd429cb4a015a5d68e277e', 'request_id': 'req-399a022d-5ec4-4b2e-ab83-7e9182799bee'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:08.070358 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a02ae0275bcd429cb4a015a5d68e277e', 'request_id': 'req-399a022d-5ec4-4b2e-ab83-7e9182799bee'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:08.076807 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a02ae0275bcd429cb4a015a5d68e277e', 'request_id': 'req-399a022d-5ec4-4b2e-ab83-7e9182799bee'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:08.087205 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] The token cache is being invalidate because project a02ae0275bcd429cb4a015a5d68e277e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:04:08.087489 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:08.088131 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:08.090288 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1458/2896] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:04:07 2022] DELETE /identity/v3/projects/a02ae0275bcd429cb4a015a5d68e277e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:04:17.717369 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:04:18.084099 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:04:24.213607 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4306bf-5705-4acf-b255-b6f60ef65983 service neutron] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:04:24.215096 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-db4306bf-5705-4acf-b255-b6f60ef65983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:04:24.228541 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4306bf-5705-4acf-b255-b6f60ef65983 service neutron] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:04:24.284921 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:04:24.285751 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:04:24.286029 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:04:24.286300 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] PATH_INFO: `/v3/users/48d92050460c411997ab955ff07dc3cd` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:04:24.287470 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48d92050460c411997ab955ff07dc3cd)` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:04:24.289378 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:04:24.335426 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48d92050460c411997ab955ff07dc3cd'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:24.348000 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48d92050460c411997ab955ff07dc3cd'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:24.355286 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48d92050460c411997ab955ff07dc3cd'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:24.362951 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48d92050460c411997ab955ff07dc3cd'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:24.363592 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.notifications [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48d92050460c411997ab955ff07dc3cd'} {{(pid=86779) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:24.369421 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1439/2897] 10.222.0.66 () {58 vars in 1223 bytes} [Fri Jul 1 20:04:24 2022] DELETE /identity/v3/users/48d92050460c411997ab955ff07dc3cd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:04:24.540161 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] Authenticating user token {{(pid=86778) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:04:24.541652 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.fernet_utils [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86778) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:04:24.556674 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a022d-5ec4-4b2e-ab83-7e9182799bee admin admin] Validating token access rules against request {{(pid=86778) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:04:24.615123 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d09bdd9927e44ffbfd9b2ec447a5887', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54a2d5d499f64a6a8437120d4d4f1f46', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86778) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:04:24.615981 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:04:24.616260 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:04:24.616534 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] PATH_INFO: `/v3/projects/8fc7896ac2f04a87bbe1e19b92c87fc6` {{(pid=86778) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:04:24.617750 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fc7896ac2f04a87bbe1e19b92c87fc6)` {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:04:24.619660 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] RBAC: Authorization granted {{(pid=86778) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:04:24.673047 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc7896ac2f04a87bbe1e19b92c87fc6', 'request_id': 'req-89293a46-8f72-4be8-8339-bfd3b1bf09fc'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:24.683873 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc7896ac2f04a87bbe1e19b92c87fc6', 'request_id': 'req-89293a46-8f72-4be8-8339-bfd3b1bf09fc'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:24.690422 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc7896ac2f04a87bbe1e19b92c87fc6', 'request_id': 'req-89293a46-8f72-4be8-8339-bfd3b1bf09fc'} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:24.701444 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] The token cache is being invalidate because project 8fc7896ac2f04a87bbe1e19b92c87fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86778) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 20:04:24.701762 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:24.702410 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG keystone.notifications [None req-89293a46-8f72-4be8-8339-bfd3b1bf09fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86778) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 20:04:24.703988 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: [pid: 86778|app: 0|req: 1459/2898] 10.222.0.66 () {58 vars in 1232 bytes} [Fri Jul 1 20:04:24 2022] DELETE /identity/v3/projects/8fc7896ac2f04a87bbe1e19b92c87fc6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 20:04:34.365948 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:04:34.698025 dv-cinder-844601-7-iscsi devstack@keystone.service[86778]: DEBUG dbcounter [-] [86778] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86778) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 20:06:55.430850 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] Authenticating user token {{(pid=86779) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 20:06:55.432416 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:06:55.576173 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3846f0-23dd-4c38-9cc6-f525d74c1009 admin admin] Validating token access rules against request {{(pid=86779) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 20:06:55.633217 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2964932-ebcf-4d2b-9651-cac5644bcca3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bba7d9205e694f2182d6091061d1219c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd62abd45e824417a0dda359d6fdabcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86779) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 20:06:55.633896 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2964932-ebcf-4d2b-9651-cac5644bcca3 service placement] REQUEST_METHOD: `GET` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 20:06:55.634175 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2964932-ebcf-4d2b-9651-cac5644bcca3 service placement] SCRIPT_NAME: `/identity` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 20:06:55.634449 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2964932-ebcf-4d2b-9651-cac5644bcca3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86779) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 20:06:55.636216 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.fernet_utils [None req-a2964932-ebcf-4d2b-9651-cac5644bcca3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86779) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 20:06:55.647284 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2964932-ebcf-4d2b-9651-cac5644bcca3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 20:06:55.648852 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2964932-ebcf-4d2b-9651-cac5644bcca3 service placement] RBAC: Authorization granted {{(pid=86779) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 20:06:55.658963 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: [pid: 86779|app: 0|req: 1440/2899] 10.222.0.66 () {60 vars in 1365 bytes} [Fri Jul 1 20:06:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 20:07:05.654735 dv-cinder-844601-7-iscsi devstack@keystone.service[86779]: DEBUG dbcounter [-] [86779] Writing DB stats keystone:SELECT=89 {{(pid=86779) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}